-- Logs begin at Fri 2022-08-12 10:08:44 UTC, end at Fri 2022-08-12 11:08:05 UTC. -- Aug 12 10:36:18.399043 dv-cinder-852945-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 12 10:36:18.409683 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 12 10:36:18 2022] *** Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: nodename: dv-cinder-852945-1-iscsi Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: machine: x86_64 Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: clock source: unix Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: pcre jit disabled Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: detected number of CPU cores: 4 Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: current working directory: / Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: detected binary path: /usr/bin/uwsgi-core Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: your processes number limit is 47680 Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: your memory page size is 4096 bytes Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: detected max file descriptor number: 2048 Aug 12 10:36:18.411705 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: lock engine: pthread robust mutexes Aug 12 10:36:18.412517 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: thunder lock: enabled Aug 12 10:36:18.412517 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 12 10:36:18.412517 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 12 10:36:18.450203 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: Python main interpreter initialized at 0x559ae85bf9c0 Aug 12 10:36:18.450203 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: python threads support enabled Aug 12 10:36:18.450203 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: your server socket listen backlog is limited to 100 connections Aug 12 10:36:18.450203 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: your mercy for graceful operations on workers is 90 seconds Aug 12 10:36:18.450569 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: mapped 403077 bytes (393 KB) for 2 cores Aug 12 10:36:18.450642 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: *** Operational MODE: preforking *** Aug 12 10:36:18.450889 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: *** uWSGI is running in multiple interpreter mode *** Aug 12 10:36:18.450889 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: spawned uWSGI master process (pid: 93125) Aug 12 10:36:18.451101 dv-cinder-852945-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 12 10:36:18.452475 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: spawned uWSGI worker 1 (pid: 93138, cores: 1) Aug 12 10:36:18.452475 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: spawned uWSGI worker 2 (pid: 93139, cores: 1) Aug 12 10:36:18.452475 dv-cinder-852945-1-iscsi devstack@keystone.service[93125]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 12 10:36:23.922536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 10:36:23.922536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: from cryptography.utils import int_from_bytes Aug 12 10:36:23.923378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 10:36:23.923378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: from cryptography.utils import int_from_bytes Aug 12 10:36:23.937360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 10:36:23.937360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: from cryptography.utils import int_from_bytes Aug 12 10:36:23.937899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 10:36:23.937899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: from cryptography.utils import int_from_bytes Aug 12 10:36:23.971777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 12 10:36:23.972000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 12 10:36:23.972243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] command line args: [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 12 10:36:23.972485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 12 10:36:23.972684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ================================================================================ {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 12 10:36:23.973031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] admin_token = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.973339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] config_dir = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.973567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.973733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] config_source = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.973970 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] debug = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.974184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.975435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.975435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.975435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.975435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.975661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] list_limit = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.975809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] log_config_append = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.976026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.976231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] log_dir = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.976449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] log_file = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.976735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.977034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.977321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.977609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.977781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.977952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.978223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.978499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.978773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.979058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.979341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.979624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.979919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.980216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.980617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.981195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.981396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] publish_errors = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.981682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.981964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.982165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.982463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.982747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.983032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] standard_threads = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.983321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.983615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.983817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.984018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] use_journal = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.984218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] use_json = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.984442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] use_stderr = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.984645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] use_syslog = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.984927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.985233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.985523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.985606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 12 10:36:23.985762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 12 10:36:23.985868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.985981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] command line args: [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 12 10:36:23.986172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.986334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 12 10:36:23.986508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.986585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ================================================================================ {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 12 10:36:23.986844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.986929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] admin_token = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.987178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] config_dir = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.987283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.987437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.987661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.987846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] config_source = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.987964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.988210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.988402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] debug = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.988482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.988691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.988807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.988934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.989009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.989164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.989241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.989404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.989577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.989657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.989864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.990025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.990108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.990339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] list_limit = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.990420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.990578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.990654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] log_config_append = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.990807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.990933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.991056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] log_dir = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.991135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.991271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] log_file = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.991347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.991527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.991613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.991769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.991916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.991992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.992205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.992326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.992454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.992599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.992686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.992819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.992919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.993039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.993167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.993352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.993429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.993602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.993690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.993841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.993960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.994078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.994279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.994356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.994533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.994620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.994757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.994901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.994978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.995200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.995274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.995422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.995557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.995633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.995854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.995967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.996082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.996308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.996521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.996600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.996756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] publish_errors = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.996911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.996988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.997057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.997191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.997393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.997496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.997614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.997746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.997858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.997936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.998080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.998251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.998343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.998522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.998599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] standard_threads = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.998748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.998866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.998974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.999194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.999276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.999413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.999494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.999654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.999734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] use_journal = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:23.999883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:23.999963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] use_json = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:24.000112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.000198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] use_stderr = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:24.000358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.000450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] use_syslog = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:24.000609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.000687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 10:36:24.000838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.001995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.002995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.003196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.003307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.003443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.003672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.003899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.003973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.004133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.004209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.004383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.004701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.004829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.004829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.005925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.006152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.006228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.006386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.006503 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.006613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.006781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.006859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.007925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.008985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.009090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.009203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.009344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.009540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.009618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.009818 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.009901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] auth.external = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.010990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] auth.password = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.011136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.011214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] auth.token = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.011433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.011510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.011660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.011772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.011896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.012096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.012181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.012370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.012451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.012611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.012751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.012828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] credential.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.013993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.014068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.014227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.014334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.014453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.014642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.014721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.014939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.015014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.015170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.015267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.015416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.015592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.015671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.015914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.016352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.016504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.016660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.016744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.016885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.016964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.017188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.017264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.017417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.017506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.017655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.017819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.017895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.018103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.018178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.018343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.018450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.018572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.018786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.018862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.019193 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.019446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.019612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.019689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.019908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.019983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.020989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021571 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.021888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.022962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.023978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.024927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.025972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.026972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.027885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.028937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.029939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] auth.external = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.030848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] auth.password = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] auth.token = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.031989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.032943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] credential.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.033858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.034008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.034088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.034235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.034314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.034514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.034595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.034739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.035928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.036953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.037892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038813 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.038964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.039972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.040923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.041993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.042989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] role.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] role.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.043932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.044920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045136 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.045979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.046911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.047897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.048988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.049935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.050108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.050372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.050616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.050735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.050827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.050901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.051909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.052932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.053940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054325 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.054914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.connection = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.055998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.056952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.057954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.058845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.059833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 12 10:36:24.059904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.060136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.060406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.060634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.060865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.061100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.061344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.061586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.061819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.062064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.062309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.062541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.062782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.063016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.063250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.063480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] role.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.063721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] role.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.063946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.064191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.064456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.064700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.064942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.065184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.065431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.065770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.066447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.066685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.066933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.067179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.067425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.067668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.068068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.068321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.068571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.068818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.069062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.069307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.069537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.069890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.070140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.070377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.070605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.070833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.071060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.071305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.071555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.071797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.072031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.072300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.072514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.072745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.072976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.073219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.073451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.073699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.073941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.074174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.074412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.074649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.074893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.075123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.075357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.075592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.075833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.076077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.076338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.076578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.076864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.077120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.connection = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.077361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.077603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.077835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.078070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.078300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.078532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.078763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.078994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.079234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.079465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.079703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.079931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.080173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.080431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.080671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.080898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.081131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.081487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 10:36:24.082059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 12 10:36:24.088616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.088845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.089065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.098791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.107422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.107654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.107869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.118254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.126961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.127192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.136901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.137133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.179486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.180315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.228668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.228861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.231370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.231604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.293437 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.293617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.293818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.293944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.294869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.317264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.317502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.317715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.317904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.318914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.319147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.329436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.329968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.342344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.342525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.342777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.342933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.345735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 10:36:24.360868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.361085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 10:36:24.361182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.361248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 10:36:24.361410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.361614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.361713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 10:36:24.361779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 10:36:24.361840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.362044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.362182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.362254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.362322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.362615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.362825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.362929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.374137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.374435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.374567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.374655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.374730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.374937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.375062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.375293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.381928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.382822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.384969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.385255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.385532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.385809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.385933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.386068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.386162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.386349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.386477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.386599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.386700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.386886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.387934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.388201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.388493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.410744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.412209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.412516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.413014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.414496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.414775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.417061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.417341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.417622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.419383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.419663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.419948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.424170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.424482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.427466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.427752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.428588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.428874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.429137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.429423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.429740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.429971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.430248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.430526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.431815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.432106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.432417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.432700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.432982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.433275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.433556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.433821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.446457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.446740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.450755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.450879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.451087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.453910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.454199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.455079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.458735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.458860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.459051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.461892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.463423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.465059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.465328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.465640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.465905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.466203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.466480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.466606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.469742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.470024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.470327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.470599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.470902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.471188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.483314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.483596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.483986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.484429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.489908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.490230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.490498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.490729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.491716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.493305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.493559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.496385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.496663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.496966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.497225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.497510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.497794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.499344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.499616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.502425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.502708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.502995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.503583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.504068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.512169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.512627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.513071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.513510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.513945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.514379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.514804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.521672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.521951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.522222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.522493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.522767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.523034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.523288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.533469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.537061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.539981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.541576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.543053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.543333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.543630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.544218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.547181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.549986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.550414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.551753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.552032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.553523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.553818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.554100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.554695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.555439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.555722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.555993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.560467 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.562031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.563978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.564297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.564569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.564945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.565228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.565507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.565873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.571566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.571849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.572130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.572425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.572692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.572969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.573239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.573491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.573858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.573945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.574134 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.574223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.574415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.574495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.574679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.574943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.575196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.581352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 10:36:24.581649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.581906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.582178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.582449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.582716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.582985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.583250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.583638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.583920 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.584187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.584468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.584740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.585010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 10:36:24.608407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559ae85bf9c0 pid: 93139 (default app) Aug 12 10:36:24.611720 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8681bd-3530-4f85-a846-7a14e9d75c6e None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:24.611999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8681bd-3530-4f85-a846-7a14e9d75c6e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:24.612295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8681bd-3530-4f85-a846-7a14e9d75c6e None None] PATH_INFO: `/v3` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:24.617469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Fri Aug 12 10:36:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:24.618405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559ae85bf9c0 pid: 93138 (default app) Aug 12 10:36:32.104752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b5279-77fc-402c-9725-87804408458c None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:32.105646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b5279-77fc-402c-9725-87804408458c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:32.105646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b5279-77fc-402c-9725-87804408458c None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:32.107632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 2/2] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:32.118185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:32.118676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:32.118676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:32.188945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: INFO dbcounter [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] Registered counter for database keystone Aug 12 10:36:32.210374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG oslo_db.sqlalchemy.engines [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93138) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 12 10:36:32.222342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writer thread running {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 12 10:36:32.415123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG passlib.handlers.bcrypt [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93138) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 12 10:36:32.415550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG passlib.handlers.bcrypt [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93138) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 12 10:36:32.472037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:32.473653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:32.623291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1/3] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 12 10:36:33.120716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b5279-77fc-402c-9725-87804408458c None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:33.190769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: INFO dbcounter [None req-0b9b5279-77fc-402c-9725-87804408458c None None] Registered counter for database keystone Aug 12 10:36:33.215584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG oslo_db.sqlalchemy.engines [None req-0b9b5279-77fc-402c-9725-87804408458c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93139) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 12 10:36:33.227658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writer thread running {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 12 10:36:33.350448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b5279-77fc-402c-9725-87804408458c None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:33.354728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896592c-191c-43e6-a18b-7ee3a2ee54a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:33.355358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896592c-191c-43e6-a18b-7ee3a2ee54a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:33.355632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896592c-191c-43e6-a18b-7ee3a2ee54a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:33.355901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896592c-191c-43e6-a18b-7ee3a2ee54a1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:33.364538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896592c-191c-43e6-a18b-7ee3a2ee54a1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:33.415870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0896592c-191c-43e6-a18b-7ee3a2ee54a1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 12 10:36:33.416205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896592c-191c-43e6-a18b-7ee3a2ee54a1 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:33.426688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 3/4] 10.222.0.23 () {58 vars in 1324 bytes} [Fri Aug 12 10:36:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:36:33.433916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:33.456786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d66122-a48e-48ce-8618-c45144ce86d3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:33.463170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3121d8-7f00-49bf-8953-ed9c8e0e70fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:33.464301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3121d8-7f00-49bf-8953-ed9c8e0e70fa admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:33.464575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3121d8-7f00-49bf-8953-ed9c8e0e70fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:33.464846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3121d8-7f00-49bf-8953-ed9c8e0e70fa admin admin] PATH_INFO: `/v3/projects/1b6e50f80fbe471b845ef361f068a613` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:33.466499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3121d8-7f00-49bf-8953-ed9c8e0e70fa admin admin] RBAC: Authorizing `identity:get_project(project_id=1b6e50f80fbe471b845ef361f068a613)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:33.516665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7b3121d8-7f00-49bf-8953-ed9c8e0e70fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 12 10:36:33.516999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3121d8-7f00-49bf-8953-ed9c8e0e70fa admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:33.521691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 2/5] 10.222.0.23 () {56 vars in 1207 bytes} [Fri Aug 12 10:36:33 2022] GET /identity/v3/projects/1b6e50f80fbe471b845ef361f068a613 => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:35.381823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebecc0-a409-450f-94b2-bc30410f9f9a None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:35.382613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebecc0-a409-450f-94b2-bc30410f9f9a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:35.382613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebecc0-a409-450f-94b2-bc30410f9f9a None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:35.386914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 4/6] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:35.394572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c4012-480e-4913-a368-62e2a41d4c4e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:35.395017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c4012-480e-4913-a368-62e2a41d4c4e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:35.395116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c4012-480e-4913-a368-62e2a41d4c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:35.417010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1f9c4012-480e-4913-a368-62e2a41d4c4e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:35.418493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1f9c4012-480e-4913-a368-62e2a41d4c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:35.427239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 3/7] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 12 10:36:35.691601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebecc0-a409-450f-94b2-bc30410f9f9a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:35.706261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebecc0-a409-450f-94b2-bc30410f9f9a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:35.711722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09385e-0f9d-4c73-8fe3-471829b72ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:35.712307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09385e-0f9d-4c73-8fe3-471829b72ecb admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:35.712613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09385e-0f9d-4c73-8fe3-471829b72ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:35.712875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09385e-0f9d-4c73-8fe3-471829b72ecb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:35.721364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09385e-0f9d-4c73-8fe3-471829b72ecb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:35.723133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09385e-0f9d-4c73-8fe3-471829b72ecb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:35.732512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 5/8] 10.222.0.23 () {58 vars in 1324 bytes} [Fri Aug 12 10:36:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:36:35.738716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c4012-480e-4913-a368-62e2a41d4c4e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:35.748214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c4012-480e-4913-a368-62e2a41d4c4e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:35.754429 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4df34e-e898-476c-97ba-2d8ed400316a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:35.755055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4df34e-e898-476c-97ba-2d8ed400316a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:35.755331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4df34e-e898-476c-97ba-2d8ed400316a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:35.755595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4df34e-e898-476c-97ba-2d8ed400316a admin admin] PATH_INFO: `/v3/users/28302e0508fb4cc195a19e42f7da3046` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:35.757091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4df34e-e898-476c-97ba-2d8ed400316a admin admin] RBAC: Authorizing `identity:get_user(user_id=28302e0508fb4cc195a19e42f7da3046)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:35.758934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4df34e-e898-476c-97ba-2d8ed400316a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:35.760491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 4/9] 10.222.0.23 () {56 vars in 1198 bytes} [Fri Aug 12 10:36:35 2022] GET /identity/v3/users/28302e0508fb4cc195a19e42f7da3046 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:37.701695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8148398-2288-4f30-ba9f-8981282ed863 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:37.702261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8148398-2288-4f30-ba9f-8981282ed863 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:37.702261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8148398-2288-4f30-ba9f-8981282ed863 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:37.703511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 6/10] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:37.709752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90598-0a60-4258-ac8f-aa22d54f6c8f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:37.710208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90598-0a60-4258-ac8f-aa22d54f6c8f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:37.710298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90598-0a60-4258-ac8f-aa22d54f6c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:37.728789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0dd90598-0a60-4258-ac8f-aa22d54f6c8f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:37.729987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0dd90598-0a60-4258-ac8f-aa22d54f6c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:37.751261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b484911-5655-484a-b211-815fe57bdd7b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:37.751510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b484911-5655-484a-b211-815fe57bdd7b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:37.751780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b484911-5655-484a-b211-815fe57bdd7b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:37.753711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 7/11] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:37.758750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:37.759275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:37.759633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:37.804792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG passlib.handlers.bcrypt [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93139) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 12 10:36:37.805197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG passlib.handlers.bcrypt [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93139) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 12 10:36:37.816116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 5/12] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:36:37.850166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:37.851778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:37.951926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 8/13] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:38.081950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd90598-0a60-4258-ac8f-aa22d54f6c8f None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.107345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:38.187490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd90598-0a60-4258-ac8f-aa22d54f6c8f None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.212095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.219976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.220623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.221113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.221461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.233671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:38.242437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:38.243392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:36:38.244200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 6/14] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:38.249367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.312298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5124e70-ae39-418b-9d6d-767fbb96b431 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.314644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:38.333231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:38.352919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.353468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f48ae8-82fe-4f0d-9e5e-2e797dd4343a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.354300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.354765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.355187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.391448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.392058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.392364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.392634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.393301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:38.393593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:38.395000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:38.406396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 7/15] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:36:38.412153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.437089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:38.438616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:38.544427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 9/16] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:36:38.550564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.553837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:38.621599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014ef0d-2fb9-4a31-ac85-e2be3fa1f619 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.653488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86546dfe-61a0-4225-b31d-a142bcaa5d62 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.660710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.661347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.661600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.662144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] PATH_INFO: `/v3/users/28302e0508fb4cc195a19e42f7da3046` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.663089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] RBAC: Authorizing `identity:get_user(user_id=28302e0508fb4cc195a19e42f7da3046)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:38.664588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:38.666171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 8/17] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/users/28302e0508fb4cc195a19e42f7da3046 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:36:38.672011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.687960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:38.693673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.694540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.694810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.695079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.713839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:38.715335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:36:38.734343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:38.735295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:36:38.736086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:36:38.736086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:36:38.736086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:38.736086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:36:38.741992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.778065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1013e51-bcda-4161-9ebe-0e5299a894af None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.800372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:38.812279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.812982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.813259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.813527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.814939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:38.816512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:38.818070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:38.824226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:38.844505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e4479-8036-4d5e-89cc-dabc82fdb5a7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.879873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4b9b6-f79d-4301-a114-a3444a9fc67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.880538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4b9b6-f79d-4301-a114-a3444a9fc67e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.880800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4b9b6-f79d-4301-a114-a3444a9fc67e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.881065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4b9b6-f79d-4301-a114-a3444a9fc67e None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.881711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4b9b6-f79d-4301-a114-a3444a9fc67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:38.882008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4b9b6-f79d-4301-a114-a3444a9fc67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:38.883399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4b9b6-f79d-4301-a114-a3444a9fc67e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:38.901075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:36:38.904727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:38.926720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0ba76-2d43-446e-919c-14c385ada8e9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:38.965081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cf20c-d698-4ddd-ba5b-b02b78b50983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:38.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cf20c-d698-4ddd-ba5b-b02b78b50983 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:38.965972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cf20c-d698-4ddd-ba5b-b02b78b50983 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:38.966238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cf20c-d698-4ddd-ba5b-b02b78b50983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:38.966974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73cf20c-d698-4ddd-ba5b-b02b78b50983 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.domain.id=default, user.id=28302e0508fb4cc195a19e42f7da3046) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:38.967273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73cf20c-d698-4ddd-ba5b-b02b78b50983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:38.968778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73cf20c-d698-4ddd-ba5b-b02b78b50983 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:38.977022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1322 bytes} [Fri Aug 12 10:36:38 2022] GET /identity/v3/role_assignments?user.id=28302e0508fb4cc195a19e42f7da3046&scope.domain.id=default&role.id=e65d667683404c438483a1b1252ff2bb => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:41.107223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1927-99f1-4bb4-8d96-a71eace20720 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.107746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1927-99f1-4bb4-8d96-a71eace20720 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.107746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1927-99f1-4bb4-8d96-a71eace20720 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.109422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:41.115228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c759ffc-9216-476b-b718-da73958d6669 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.115650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c759ffc-9216-476b-b718-da73958d6669 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.115834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c759ffc-9216-476b-b718-da73958d6669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.139201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8c759ffc-9216-476b-b718-da73958d6669 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:41.140730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8c759ffc-9216-476b-b718-da73958d6669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:41.234707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:36:41.296209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037e1ca-014f-44dc-962b-55661fec6467 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.296542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037e1ca-014f-44dc-962b-55661fec6467 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.296793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037e1ca-014f-44dc-962b-55661fec6467 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.298275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:41.303720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5192-2413-4e91-b52a-acc06744057b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.304009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5192-2413-4e91-b52a-acc06744057b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.304303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5192-2413-4e91-b52a-acc06744057b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.333402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-515b5192-2413-4e91-b52a-acc06744057b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:41.334683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-515b5192-2413-4e91-b52a-acc06744057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:41.433921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 12/25] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:41.612095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095cbc7-c241-4912-8c8b-13ca555928ad None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.612615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095cbc7-c241-4912-8c8b-13ca555928ad None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.613027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095cbc7-c241-4912-8c8b-13ca555928ad None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.614546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 14/26] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:41.626023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5ca9-30b7-493c-a943-7408395d1fef None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.626311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5ca9-30b7-493c-a943-7408395d1fef None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.626587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5ca9-30b7-493c-a943-7408395d1fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.641466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:41.657307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7ebc5ca9-30b7-493c-a943-7408395d1fef None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:41.658585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7ebc5ca9-30b7-493c-a943-7408395d1fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:41.704159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095cbc7-c241-4912-8c8b-13ca555928ad None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:41.770465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 13/27] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:41.771742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877c50-e229-40ed-b659-2b780e395e2d None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.772022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877c50-e229-40ed-b659-2b780e395e2d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.777078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c877c50-e229-40ed-b659-2b780e395e2d None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.778585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 14/28] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:41.784461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1717-1902-4421-b9b7-81628f9f9b7c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.784740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1717-1902-4421-b9b7-81628f9f9b7c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.785010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1717-1902-4421-b9b7-81628f9f9b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:41.824093 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-951d1717-1902-4421-b9b7-81628f9f9b7c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:41.825341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-951d1717-1902-4421-b9b7-81628f9f9b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:41.917847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095cbc7-c241-4912-8c8b-13ca555928ad None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:41.955782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:41.961864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 15/29] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:41.962933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d1717-1902-4421-b9b7-81628f9f9b7c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:41.988733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:41.989714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:41.991027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:41.992099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.016773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:42.101505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:42.158112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:42.159758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:42.160747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:42.162719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 15/30] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:36:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:42.163651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:42.184442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d1717-1902-4421-b9b7-81628f9f9b7c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:42.223119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:42.224063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.224366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.224631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.225550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:42.227220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:42.354239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66e61-861b-450b-983c-5c484e842681 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:42.378901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 16/31] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Aug 12 10:36:41 2022] POST /identity/v3/roles => generated 219 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:42.379925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:42.421819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:42.422462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.422738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.423005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.435383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:42.437281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:36:42.467708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:42.468774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:36:42.469551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:36:42.469551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:36:42.469551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:42.472194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 16/32] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:42.477707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:42.522595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:42.536594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7472b59-398f-491c-9511-c7e81e1a5279 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:42.566547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:42.583376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66094a5a-e9aa-4017-8604-7efc09ae785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:42.583996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094a5a-e9aa-4017-8604-7efc09ae785b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.584347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094a5a-e9aa-4017-8604-7efc09ae785b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.584573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094a5a-e9aa-4017-8604-7efc09ae785b None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.585254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66094a5a-e9aa-4017-8604-7efc09ae785b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:42.585557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66094a5a-e9aa-4017-8604-7efc09ae785b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:42.587097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66094a5a-e9aa-4017-8604-7efc09ae785b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:42.599116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 17/33] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:42.600740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4a63-1ab8-40ea-b0a4-60dca0e50760 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.601023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4a63-1ab8-40ea-b0a4-60dca0e50760 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.601299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e4a63-1ab8-40ea-b0a4-60dca0e50760 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.602974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 18/34] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:42.603825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e4a63-1ab8-40ea-b0a4-60dca0e50760 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:42.615719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506634b-5c88-447e-b6b9-50d3b012b28e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:42.663227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90a605-30de-4711-9171-346e2614c93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:42.663847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90a605-30de-4711-9171-346e2614c93a None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.664612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90a605-30de-4711-9171-346e2614c93a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.664612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90a605-30de-4711-9171-346e2614c93a None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.665234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90a605-30de-4711-9171-346e2614c93a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:42.666772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90a605-30de-4711-9171-346e2614c93a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:42.733308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e4a63-1ab8-40ea-b0a4-60dca0e50760 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:42.770439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:42.771451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.771899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.772434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.773307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:42.773778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:42.775339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:42.793812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 19/35] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:42.794891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:42.797652 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 17/36] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Aug 12 10:36:42 2022] POST /identity/v3/roles => generated 225 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:42.799552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e785fa-f08a-4d1a-b7ba-04ee85bbb12e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.799841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e785fa-f08a-4d1a-b7ba-04ee85bbb12e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.800116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e785fa-f08a-4d1a-b7ba-04ee85bbb12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.819839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d3e785fa-f08a-4d1a-b7ba-04ee85bbb12e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:42.821500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d3e785fa-f08a-4d1a-b7ba-04ee85bbb12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:42.864064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:42.898862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 18/37] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:42.904463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccc617-5e69-4e98-a259-3e8d8e324ce9 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.904463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccc617-5e69-4e98-a259-3e8d8e324ce9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.904463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccc617-5e69-4e98-a259-3e8d8e324ce9 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.904463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 19/38] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:42.904463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccc617-5e69-4e98-a259-3e8d8e324ce9 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:42.925339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:42.944837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065cbcf-8dca-45ff-98d3-24fad20f0b60 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:42.988056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671f9a8-345b-4396-ba3e-2ba441968573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:42.988652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671f9a8-345b-4396-ba3e-2ba441968573 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.988932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671f9a8-345b-4396-ba3e-2ba441968573 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.989197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671f9a8-345b-4396-ba3e-2ba441968573 None admin] PATH_INFO: `/v3/users/28302e0508fb4cc195a19e42f7da3046` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:42.990403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671f9a8-345b-4396-ba3e-2ba441968573 None admin] RBAC: Authorizing `identity:get_user(user_id=28302e0508fb4cc195a19e42f7da3046)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:42.991857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671f9a8-345b-4396-ba3e-2ba441968573 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:42.994356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 20/39] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:42 2022] GET /identity/v3/users/28302e0508fb4cc195a19e42f7da3046 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:42.994715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142d7c6-08a0-4119-aa6a-9c6c4fb39656 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:42.994994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142d7c6-08a0-4119-aa6a-9c6c4fb39656 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:42.995260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142d7c6-08a0-4119-aa6a-9c6c4fb39656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:43.016513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6142d7c6-08a0-4119-aa6a-9c6c4fb39656 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:43.019209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6142d7c6-08a0-4119-aa6a-9c6c4fb39656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:43.043825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccc617-5e69-4e98-a259-3e8d8e324ce9 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:43.081766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:43.082451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:43.082730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:43.082999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:43.083893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:43.085641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:43.097107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 21/40] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:43.105189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142d7c6-08a0-4119-aa6a-9c6c4fb39656 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:43.120643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 20/41] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:36:42 2022] POST /identity/v3/projects => generated 301 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:43.166926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:43.186253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:43.234103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142d7c6-08a0-4119-aa6a-9c6c4fb39656 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:43.277675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:43.289433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:43.289683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:43.290020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:43.290294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:43.291477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:43.293453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:43.296129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 22/42] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:43.311062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:43.322053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b65fe-8b19-4d21-8eb5-05a9223cbc09 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:43.363879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:43.364542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:43.364813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:43.365082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:43.365743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:43.367138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:43.385647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 21/43] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Aug 12 10:36:43 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:36:43.386529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:43.433160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32254b-5f47-4783-8c64-f91f620c324f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:43.448340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:43.475744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:43.476455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:43.476732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:43.477006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] PATH_INFO: `/v3/domains/default/users/28302e0508fb4cc195a19e42f7da3046/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:43.479105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=28302e0508fb4cc195a19e42f7da3046, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:43.480566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:43.497040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 23/44] 10.222.0.23 () {58 vars in 1361 bytes} [Fri Aug 12 10:36:43 2022] PUT /identity/v3/domains/default/users/28302e0508fb4cc195a19e42f7da3046/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:36:43.505625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c446fe1-c598-46e3-9340-1d2a59a55bfc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:43.542035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:43.550885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b139360-a920-45b6-9b48-6c4fce3628dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:43.552196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139360-a920-45b6-9b48-6c4fce3628dc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:43.552573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139360-a920-45b6-9b48-6c4fce3628dc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:43.552848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139360-a920-45b6-9b48-6c4fce3628dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:43.555066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b139360-a920-45b6-9b48-6c4fce3628dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:43.556758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b139360-a920-45b6-9b48-6c4fce3628dc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:43.558686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 22/45] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:43.564896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:43.670266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ee146-b719-4031-be2f-09d0db31cadb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:43.704292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58d082-901d-47e5-8da1-3323465fe506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:43.704968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d082-901d-47e5-8da1-3323465fe506 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:43.705234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d082-901d-47e5-8da1-3323465fe506 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:43.705506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d082-901d-47e5-8da1-3323465fe506 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:43.706153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58d082-901d-47e5-8da1-3323465fe506 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:43.707598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58d082-901d-47e5-8da1-3323465fe506 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:43.737241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 24/46] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:36:43 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:45.367577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25d97a-03e6-4426-8ca8-e2eefc88c8d4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:45.368078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25d97a-03e6-4426-8ca8-e2eefc88c8d4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:45.368078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25d97a-03e6-4426-8ca8-e2eefc88c8d4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:45.369502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 23/47] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:45.374816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e45b08-ce9c-4544-855c-b0967bcb1457 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:45.375226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e45b08-ce9c-4544-855c-b0967bcb1457 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:45.375360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e45b08-ce9c-4544-855c-b0967bcb1457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:45.394490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-e1e45b08-ce9c-4544-855c-b0967bcb1457 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:45.395682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e1e45b08-ce9c-4544-855c-b0967bcb1457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:45.413320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:45.465662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 25/48] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:45.612932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487934-98ee-4af8-ae45-2a44c011f0f0 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:45.613112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487934-98ee-4af8-ae45-2a44c011f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:45.613389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487934-98ee-4af8-ae45-2a44c011f0f0 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:45.614717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 24/49] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:45.620005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07bac4-6c09-4031-af41-1683a8b03fd9 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:45.620317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07bac4-6c09-4031-af41-1683a8b03fd9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:45.620593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07bac4-6c09-4031-af41-1683a8b03fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:45.640530 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-af07bac4-6c09-4031-af41-1683a8b03fd9 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:45.641715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-af07bac4-6c09-4031-af41-1683a8b03fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:45.712559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 26/50] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:45.728095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487934-98ee-4af8-ae45-2a44c011f0f0 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:45.834801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487934-98ee-4af8-ae45-2a44c011f0f0 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:45.868822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:45.869369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:45.869645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:45.869913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:45.879971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:45.884710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:45.885618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:45.886346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 25/51] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:36:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:45.892051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07bac4-6c09-4031-af41-1683a8b03fd9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:45.975414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:45.985205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:45.991987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07bac4-6c09-4031-af41-1683a8b03fd9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:46.026984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:46.027595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:46.027874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:46.028141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:46.028836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:46.029138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:46.030531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:46.038970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 27/52] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:46.044463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:46.049106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:46.090273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08798ad-f989-49f5-83f6-1a7b565e392d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:46.123843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:46.124436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:46.124709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:46.124972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:46.126077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:46.127728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:46.129321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 26/53] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:46.134829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:46.143260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36399d-31e0-4ad7-8664-63ef326fb7c3 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:46.175988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:46.176646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:46.176928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:46.177193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] PATH_INFO: `/v3/users/28302e0508fb4cc195a19e42f7da3046` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:46.178327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] RBAC: Authorizing `identity:get_user(user_id=28302e0508fb4cc195a19e42f7da3046)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:46.179799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:46.181393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 28/54] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:46 2022] GET /identity/v3/users/28302e0508fb4cc195a19e42f7da3046 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:36:46.186819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:46.201871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:46.232727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2b56-3196-4771-9a39-1298626a87a8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:46.257807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:46.267935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:46.268914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:46.269333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:46.269606 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:46.270260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:46.271718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:46.285675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09cad-fcde-4f12-b9bb-b609256bc00e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:46.299950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 27/55] 10.222.0.23 () {60 vars in 1140 bytes} [Fri Aug 12 10:36:46 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:46.323469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d3fd0-5297-44d5-91f2-0cd2f478e447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:46.324124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3fd0-5297-44d5-91f2-0cd2f478e447 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:46.324437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3fd0-5297-44d5-91f2-0cd2f478e447 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:46.324708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d3fd0-5297-44d5-91f2-0cd2f478e447 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:46.325908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d3fd0-5297-44d5-91f2-0cd2f478e447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:46.327395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d3fd0-5297-44d5-91f2-0cd2f478e447 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:46.329054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 29/56] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:36:46.336030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:46.449047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287a69-9cfa-4b7c-8693-0dec1a694291 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:46.484898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04719a67-1f7f-49d7-93f4-3e25580c0a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:46.485508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04719a67-1f7f-49d7-93f4-3e25580c0a39 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:46.485940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04719a67-1f7f-49d7-93f4-3e25580c0a39 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:46.486606 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04719a67-1f7f-49d7-93f4-3e25580c0a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:46.487338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04719a67-1f7f-49d7-93f4-3e25580c0a39 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.domain.id=default, user.id=28302e0508fb4cc195a19e42f7da3046) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:46.487647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04719a67-1f7f-49d7-93f4-3e25580c0a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:46.489131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04719a67-1f7f-49d7-93f4-3e25580c0a39 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:46.497935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 28/57] 10.222.0.23 () {56 vars in 1322 bytes} [Fri Aug 12 10:36:46 2022] GET /identity/v3/role_assignments?user.id=28302e0508fb4cc195a19e42f7da3046&scope.domain.id=default&role.id=e65d667683404c438483a1b1252ff2bb => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:48.181997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179e188-2af9-47c1-bf76-1e6b462bdd20 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:48.182866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179e188-2af9-47c1-bf76-1e6b462bdd20 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:48.182866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179e188-2af9-47c1-bf76-1e6b462bdd20 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:48.184995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 30/58] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:48.190414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3806df-f871-40c0-b2d1-5f4ed4f9690c None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:48.190821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3806df-f871-40c0-b2d1-5f4ed4f9690c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:48.190929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3806df-f871-40c0-b2d1-5f4ed4f9690c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:48.210166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-6f3806df-f871-40c0-b2d1-5f4ed4f9690c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:48.211338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6f3806df-f871-40c0-b2d1-5f4ed4f9690c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:48.248118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:48.281857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 29/59] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:48.544914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179e188-2af9-47c1-bf76-1e6b462bdd20 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:48.649629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:48.671774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179e188-2af9-47c1-bf76-1e6b462bdd20 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:48.709021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b61d7-79b5-4214-bfc5-39942e9440d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:48.709639 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b61d7-79b5-4214-bfc5-39942e9440d0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:48.709915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b61d7-79b5-4214-bfc5-39942e9440d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:48.710182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b61d7-79b5-4214-bfc5-39942e9440d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:48.711428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b61d7-79b5-4214-bfc5-39942e9440d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:48.712889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b61d7-79b5-4214-bfc5-39942e9440d0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:48.714460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 31/60] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:48.721370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3806df-f871-40c0-b2d1-5f4ed4f9690c None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:48.835735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3806df-f871-40c0-b2d1-5f4ed4f9690c None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:48.872807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a37a4-2e72-4c97-a585-8325159a2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:48.873390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a37a4-2e72-4c97-a585-8325159a2f7d None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:48.873664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a37a4-2e72-4c97-a585-8325159a2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:48.873933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a37a4-2e72-4c97-a585-8325159a2f7d None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:48.874568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a37a4-2e72-4c97-a585-8325159a2f7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:48.876008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a37a4-2e72-4c97-a585-8325159a2f7d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:48.907649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 30/61] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:36:48 2022] POST /identity/v3/users => generated 275 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:51.956428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368bd1-5223-4454-86e2-e44589fb050b None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:51.956964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368bd1-5223-4454-86e2-e44589fb050b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:51.956964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368bd1-5223-4454-86e2-e44589fb050b None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:51.958315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 32/62] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:51.965734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6854a-06a3-4b21-bea0-1f97acbe9107 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:51.966156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6854a-06a3-4b21-bea0-1f97acbe9107 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:51.966428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6854a-06a3-4b21-bea0-1f97acbe9107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:51.989227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-00d6854a-06a3-4b21-bea0-1f97acbe9107 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:51.990424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-00d6854a-06a3-4b21-bea0-1f97acbe9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:52.014044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:52.074160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 31/63] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:52.118460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c606f2-28c1-4f8c-821b-747e14d412ee None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.119095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c606f2-28c1-4f8c-821b-747e14d412ee None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.119561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c606f2-28c1-4f8c-821b-747e14d412ee None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.121137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 33/64] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:52.126646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f326b73-3248-4bfb-8a3c-6ac551a5a827 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.126931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f326b73-3248-4bfb-8a3c-6ac551a5a827 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.127201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f326b73-3248-4bfb-8a3c-6ac551a5a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.154223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8f326b73-3248-4bfb-8a3c-6ac551a5a827 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:52.155508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8f326b73-3248-4bfb-8a3c-6ac551a5a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:52.227409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda8449-8653-4ecb-acac-dca04acbca81 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.227896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda8449-8653-4ecb-acac-dca04acbca81 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.228366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda8449-8653-4ecb-acac-dca04acbca81 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.229845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 34/65] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:36:52.235017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559d455-a728-420a-92ec-7c5ccce10703 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.235137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 32/66] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:52.235774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559d455-a728-420a-92ec-7c5ccce10703 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.236299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559d455-a728-420a-92ec-7c5ccce10703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.273262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6559d455-a728-420a-92ec-7c5ccce10703 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:52.274810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6559d455-a728-420a-92ec-7c5ccce10703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:52.369370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f326b73-3248-4bfb-8a3c-6ac551a5a827 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:52.371281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 35/67] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:36:52.512869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:52.528157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f326b73-3248-4bfb-8a3c-6ac551a5a827 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:52.575759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36129465-f2e6-4e8c-bc71-bb729f19dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:52.576427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129465-f2e6-4e8c-bc71-bb729f19dae2 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.576707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129465-f2e6-4e8c-bc71-bb729f19dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.577050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129465-f2e6-4e8c-bc71-bb729f19dae2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.592478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36129465-f2e6-4e8c-bc71-bb729f19dae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-36129465-f2e6-4e8c-bc71-bb729f19dae2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:52.597340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:52.598263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:52.599008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 33/68] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:52.604478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559d455-a728-420a-92ec-7c5ccce10703 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:52.645899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b46e5-22e6-42c5-bf15-d7934ba31dcf None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.646176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b46e5-22e6-42c5-bf15-d7934ba31dcf None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.646604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b46e5-22e6-42c5-bf15-d7934ba31dcf None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.648076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 34/69] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:52.653326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39377a11-4985-4bdd-a8d1-4b4fc50c8de0 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.653757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39377a11-4985-4bdd-a8d1-4b4fc50c8de0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.654189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39377a11-4985-4bdd-a8d1-4b4fc50c8de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.676432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-39377a11-4985-4bdd-a8d1-4b4fc50c8de0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:52.677633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-39377a11-4985-4bdd-a8d1-4b4fc50c8de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:52.708101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:52.730690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559d455-a728-420a-92ec-7c5ccce10703 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:52.750874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 35/70] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:36:52.753849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e982e8-08f1-45d4-90a2-da03ed7d5954 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.754732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e982e8-08f1-45d4-90a2-da03ed7d5954 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.755457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e982e8-08f1-45d4-90a2-da03ed7d5954 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.757855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 36/71] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:52.759086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e982e8-08f1-45d4-90a2-da03ed7d5954 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:52.772769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39156fb-b6b0-459a-9bb6-8c3567c0083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:52.773427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39156fb-b6b0-459a-9bb6-8c3567c0083d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.773707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39156fb-b6b0-459a-9bb6-8c3567c0083d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.774085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39156fb-b6b0-459a-9bb6-8c3567c0083d None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.774655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39156fb-b6b0-459a-9bb6-8c3567c0083d None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:52.774957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39156fb-b6b0-459a-9bb6-8c3567c0083d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:52.776422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39156fb-b6b0-459a-9bb6-8c3567c0083d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:52.785010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 36/72] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:36:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:52.786062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab32eb-2a63-42b8-9d38-b97942be742e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.786356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab32eb-2a63-42b8-9d38-b97942be742e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.786631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab32eb-2a63-42b8-9d38-b97942be742e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.804206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-75ab32eb-2a63-42b8-9d38-b97942be742e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:52.805411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-75ab32eb-2a63-42b8-9d38-b97942be742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:52.852385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:52.875650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 37/73] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:52.876390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab32eb-2a63-42b8-9d38-b97942be742e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:52.885742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e982e8-08f1-45d4-90a2-da03ed7d5954 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:52.933593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:52.934743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:52.935222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:52.935909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:52.948523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:52.950496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:52.957073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:36:52.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:52.960142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 37/74] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:36:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:52.961189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:52.985826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab32eb-2a63-42b8-9d38-b97942be742e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.028305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.028858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.029131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.029400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.040495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:53.045224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:53.046097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:36:53.046850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 38/75] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:53.047467 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.070671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb018-dc71-4993-8122-7c23c9c5974b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.104653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.105284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.105556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.105821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.114857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.125038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.127143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.134116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 38/76] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:52 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.134116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.153427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecee935-0e2f-4e0e-80e1-909e35b112ec None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.185221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.191019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.191718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.192001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.192288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.192979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:53.193280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.194692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.203176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 39/77] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.204127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.233107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707216bf-5eaf-4dc7-8773-042b88f74259 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.267432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.268056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.268360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.268628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.278774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:53.283518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:53.284378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:53.285151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 39/78] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:53.285775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.315358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd190ee8-65cc-4105-8861-422a798f99ea None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.341828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.351965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.352618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.352892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.353153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.353818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:53.354112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.355484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.363900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 40/79] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.364744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.391908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c76072-0954-488c-92c4-cf592e4d37bc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.420387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.426858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.427468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.427741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.428003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.462898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43719dad-42fa-419f-bbfe-a164d3daaff6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.494766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.496343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.497331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.497950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.498225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.498490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.499591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.501229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.504498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 41/80] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.505256 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.560218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.604272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 40/81] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:36:53.604448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8ecf-0c65-4dd3-8ed7-51dee31105b8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.609221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.627088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.638075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.638701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.638978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.639595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.640863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.642806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.644420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 42/82] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.645184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.709694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b4df-2875-44ad-963d-628730262c73 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.743936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ee5f3-4961-4f19-98b4-65454fe41cae None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.746291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.746899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.747179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.747452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.748115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:53.748453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.749849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.761684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 41/83] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.762458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.781449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.782685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.783416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.783798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.784145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] PATH_INFO: `/v3/users/28302e0508fb4cc195a19e42f7da3046` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.785387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] RBAC: Authorizing `identity:get_user(user_id=28302e0508fb4cc195a19e42f7da3046)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.786849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.788660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 43/84] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/users/28302e0508fb4cc195a19e42f7da3046 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.789272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.846881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:53.864082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd486cc-0d89-4396-8dba-26e1a653cecd None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.890161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca05259-3f99-4899-b26a-7581fc27422f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:53.897872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.898475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.898748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.899024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.899663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.901148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.922951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 42/85] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:36:53 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:53.923946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:53.934584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:53.935678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:53.935934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:53.936381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:53.938397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:53.940110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:53.942935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 44/86] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:53.943575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:54.014897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:54.051549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0642a-2b29-4284-8aef-7332967b69a6 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:54.058862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea546-edde-4d44-af8b-2f365e028776 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:54.088071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:54.095838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:54.096481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:54.096758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:54.097070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:54.098853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:54.099490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:54.099771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:54.100042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:54.101271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:54.102748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:54.104357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 45/87] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:54.105664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:54.181631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:54.184075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:54.231018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989421ed-d350-4a6a-8c9a-42961abb6850 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:54.280606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:54.281580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:54.282051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:54.282557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:54.284202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:54.321615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 43/88] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:36:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:36:54.322301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:54.362919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:54.375159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:54.378273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:54.444174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7f7df-1795-465c-8837-9fc60159636c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:54.479969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:54.480620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:54.480895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:54.481163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:54.481894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=a4740f00cc764a3ca9358d80c6fb5630) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:54.482201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:54.483656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:54.496449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 44/89] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:36:54 2022] GET /identity/v3/role_assignments?user.id=a4740f00cc764a3ca9358d80c6fb5630&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:54.497598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:54.508663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 46/90] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:36:54.509895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:54.613660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:54.631185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09799e-8062-4341-94e8-1c6205b16e04 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:54.648656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:54.673095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440537b-8aab-48f2-a807-191582924117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:54.674284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440537b-8aab-48f2-a807-191582924117 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:54.674760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440537b-8aab-48f2-a807-191582924117 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:54.675226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440537b-8aab-48f2-a807-191582924117 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:54.676734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440537b-8aab-48f2-a807-191582924117 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:54.679102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440537b-8aab-48f2-a807-191582924117 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:54.681581 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 47/91] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:54 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:54.682611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440537b-8aab-48f2-a807-191582924117 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:54.756417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a628-eb2c-4c8b-8452-413b76b72df1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:54.809216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440537b-8aab-48f2-a807-191582924117 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:54.842350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:54.843758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:54.844840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:54.845793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:54.848210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31047614-296a-4569-8334-429e9e5d465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:54.850699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047614-296a-4569-8334-429e9e5d465d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:54.851175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047614-296a-4569-8334-429e9e5d465d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:54.851640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047614-296a-4569-8334-429e9e5d465d None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:54.853825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31047614-296a-4569-8334-429e9e5d465d None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:54.861057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31047614-296a-4569-8334-429e9e5d465d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:54.863382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 48/92] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:54 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:54.864589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31047614-296a-4569-8334-429e9e5d465d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:54.937237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:54.942983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:54.944800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:54.989428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:55.060547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 45/93] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:36:55.062116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:55.099662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31047614-296a-4569-8334-429e9e5d465d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:55.160539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:55.161163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:55.161507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:55.161783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:55.162632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] RBAC: Adding query filter params (role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=a4740f00cc764a3ca9358d80c6fb5630) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:55.162938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:55.164540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:55.179214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 49/94] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:36:54 2022] GET /identity/v3/role_assignments?user.id=a4740f00cc764a3ca9358d80c6fb5630&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:55.180153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:55.220725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc238fc-6889-4ffc-993e-4147bc06f9c5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:55.261771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:55.262805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:55.263198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:55.263568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:55.264752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=28302e0508fb4cc195a19e42f7da3046) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:55.265159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:55.267498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:55.275370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 46/95] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:36:55 2022] GET /identity/v3/role_assignments?user.id=28302e0508fb4cc195a19e42f7da3046&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:55.362901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33fca3-d71b-446b-9814-33b92e1d21d7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:55.373770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:55.413824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d3361-798a-4324-a256-f338d4f56398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:55.414395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d3361-798a-4324-a256-f338d4f56398 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:55.414674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d3361-798a-4324-a256-f338d4f56398 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:55.414947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d3361-798a-4324-a256-f338d4f56398 None admin] PATH_INFO: `/v3/projects/33ae5df6b7d644ed85ddd9b056650998` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:55.420515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d3361-798a-4324-a256-f338d4f56398 None admin] RBAC: Authorizing `identity:get_project(project_id=33ae5df6b7d644ed85ddd9b056650998)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:55.422042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d3361-798a-4324-a256-f338d4f56398 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:55.424362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 50/96] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:55 2022] GET /identity/v3/projects/33ae5df6b7d644ed85ddd9b056650998 => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:55.430936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:55.434481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:55.592110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466f80-0e05-484b-903c-3014acd51f43 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:55.635005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc61d2-7f98-485e-b8bb-2a4cd53fa1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:55.635617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc61d2-7f98-485e-b8bb-2a4cd53fa1b4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:55.635895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc61d2-7f98-485e-b8bb-2a4cd53fa1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:55.636168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc61d2-7f98-485e-b8bb-2a4cd53fa1b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:55.636928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc61d2-7f98-485e-b8bb-2a4cd53fa1b4 None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=33ae5df6b7d644ed85ddd9b056650998, user.id=a4740f00cc764a3ca9358d80c6fb5630) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:55.637235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc61d2-7f98-485e-b8bb-2a4cd53fa1b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:55.638674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc61d2-7f98-485e-b8bb-2a4cd53fa1b4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:55.645048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 47/97] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:36:55 2022] GET /identity/v3/role_assignments?user.id=a4740f00cc764a3ca9358d80c6fb5630&scope.project.id=33ae5df6b7d644ed85ddd9b056650998&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:56.378770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc90bec-1668-4f7b-8bfb-2c2e06ab5349 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:56.379304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc90bec-1668-4f7b-8bfb-2c2e06ab5349 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:56.379304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc90bec-1668-4f7b-8bfb-2c2e06ab5349 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:56.380680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 51/98] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:56.385780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4566b-1b53-4d25-9f4f-3383eb3b27d1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:56.386180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4566b-1b53-4d25-9f4f-3383eb3b27d1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:56.386356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4566b-1b53-4d25-9f4f-3383eb3b27d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:56.408354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-27c4566b-1b53-4d25-9f4f-3383eb3b27d1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:56.409558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-27c4566b-1b53-4d25-9f4f-3383eb3b27d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:56.484758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:56.495546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 48/99] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:56.771413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e555ad-199b-4fff-a051-64964d005193 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:56.771806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e555ad-199b-4fff-a051-64964d005193 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:56.771974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e555ad-199b-4fff-a051-64964d005193 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:56.773839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 52/100] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:56.779178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e27c95-64db-4e76-a8a2-9b3c7be6cbf4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:56.779483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e27c95-64db-4e76-a8a2-9b3c7be6cbf4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:56.779740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e27c95-64db-4e76-a8a2-9b3c7be6cbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:56.798214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-25e27c95-64db-4e76-a8a2-9b3c7be6cbf4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:56.799399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-25e27c95-64db-4e76-a8a2-9b3c7be6cbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:56.879034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 49/101] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:57.039037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e555ad-199b-4fff-a051-64964d005193 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:57.145008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b330f-4b4a-43ef-a548-8d14046b87e1 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.145322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b330f-4b4a-43ef-a548-8d14046b87e1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.145554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b330f-4b4a-43ef-a548-8d14046b87e1 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.147143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 50/102] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:57.152070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cb7f-6266-40d8-b79f-461528153bd5 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.152391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cb7f-6266-40d8-b79f-461528153bd5 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.152663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cb7f-6266-40d8-b79f-461528153bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.165018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e555ad-199b-4fff-a051-64964d005193 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:57.189299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3a85cb7f-6266-40d8-b79f-461528153bd5 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:57.190492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3a85cb7f-6266-40d8-b79f-461528153bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:57.216019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:57.216684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.216962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.217234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.238362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:57.243789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:57.244615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:36:57.245377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 53/103] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:57.253282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:57.303911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:57.390189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 51/104] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:36:57.420472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85cb7f-6266-40d8-b79f-461528153bd5 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:57.425023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39299b0-9921-4a23-b1ac-6e9b6575e646 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:57.475686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:57.476671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.477154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.477609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.478458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] RBAC: Adding query filter params (name=reader) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:57.478961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:57.480658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:57.499579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 54/105] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:36:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:57.511172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:57.543703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:57.639748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85cb7f-6266-40d8-b79f-461528153bd5 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:57.661880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:57.677869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfdf07-76ef-49dc-ba4d-5446e37a924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:57.678870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfdf07-76ef-49dc-ba4d-5446e37a924d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.679334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfdf07-76ef-49dc-ba4d-5446e37a924d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.679785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfdf07-76ef-49dc-ba4d-5446e37a924d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.681268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfdf07-76ef-49dc-ba4d-5446e37a924d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:57.682901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfdf07-76ef-49dc-ba4d-5446e37a924d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:57.684741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 52/106] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:36:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:57.686021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092f79-5ff1-42b0-b852-16acdaa47a99 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.686591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092f79-5ff1-42b0-b852-16acdaa47a99 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.687157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092f79-5ff1-42b0-b852-16acdaa47a99 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.688639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 53/107] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:57.696894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc047-7ff3-44b5-9e99-fc0a3d09e0ea None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.697421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc047-7ff3-44b5-9e99-fc0a3d09e0ea None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.697876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc047-7ff3-44b5-9e99-fc0a3d09e0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.697972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf455c-cb3b-45bf-83ee-17a58002527f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:57.736002 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:57.736938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3acfc047-7ff3-44b5-9e99-fc0a3d09e0ea None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:57.738340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3acfc047-7ff3-44b5-9e99-fc0a3d09e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:57.741423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.741982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.742444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] PATH_INFO: `/v3/users/2a44996d1c774bbbab4566578c2fe8c2` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.771403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] RBAC: Authorizing `identity:get_user(user_id=2a44996d1c774bbbab4566578c2fe8c2)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:57.773235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:57.776634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 55/108] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:57 2022] GET /identity/v3/users/2a44996d1c774bbbab4566578c2fe8c2 => generated 289 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:57.780250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:57.835659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 54/109] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:36:57.836868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfc047-7ff3-44b5-9e99-fc0a3d09e0ea None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:57.920408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2504b-aaee-4c13-9f74-561e18e8203c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:57.925181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:57.967299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:57.967914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:57.968219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:57.968990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:57.969690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:57.971671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:57.975217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfc047-7ff3-44b5-9e99-fc0a3d09e0ea None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:57.991491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:58.012722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 56/110] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:36:57 2022] POST /identity/v3/projects => generated 302 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:36:58.013669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.017212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e23a1-ddbd-4d8b-aace-a36017857ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.019108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e23a1-ddbd-4d8b-aace-a36017857ffb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.019650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e23a1-ddbd-4d8b-aace-a36017857ffb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.020218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e23a1-ddbd-4d8b-aace-a36017857ffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.120839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e23a1-ddbd-4d8b-aace-a36017857ffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:58.120839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e23a1-ddbd-4d8b-aace-a36017857ffb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:58.197768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c839d3-83a6-49ca-984d-32459fa373e6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.218899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:58.246916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eef488-66de-481c-bb76-5290a75e9a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.247546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eef488-66de-481c-bb76-5290a75e9a18 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.247820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eef488-66de-481c-bb76-5290a75e9a18 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.248087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eef488-66de-481c-bb76-5290a75e9a18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.259196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eef488-66de-481c-bb76-5290a75e9a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-79eef488-66de-481c-bb76-5290a75e9a18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:58.265209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:58.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:36:58.266875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 57/111] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:58.267595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19d3b7-e836-4b9d-a547-92df865f4fa6 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.268204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19d3b7-e836-4b9d-a547-92df865f4fa6 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.268204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19d3b7-e836-4b9d-a547-92df865f4fa6 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.269719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 58/112] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:58.270525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19d3b7-e836-4b9d-a547-92df865f4fa6 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.274588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 55/113] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:36:58.276196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978418f-b0d6-4958-9d38-abb61979d0b8 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.276595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978418f-b0d6-4958-9d38-abb61979d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.276959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978418f-b0d6-4958-9d38-abb61979d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.297754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f978418f-b0d6-4958-9d38-abb61979d0b8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:58.299240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f978418f-b0d6-4958-9d38-abb61979d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:58.340433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:58.394295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19d3b7-e836-4b9d-a547-92df865f4fa6 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.398019 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 56/114] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:58.399471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978418f-b0d6-4958-9d38-abb61979d0b8 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.428397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.429076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.429287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.429554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.443471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:58.448124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:58.448969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:36:58.449720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 59/115] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:58.450348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.516975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:58.544995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978418f-b0d6-4958-9d38-abb61979d0b8 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.553947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14770903-723d-4e5e-98c4-828130ce5e43 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.571538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:58.590454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.591079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.591351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.591617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.592769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:58.594276 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:58.596003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 60/116] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:58.597358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.617155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.618150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.618664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.619162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.626154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:58.626828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:58.628790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:58.647816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 57/117] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:58.649128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.706989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e54a4f-632c-4eb8-85bd-fd878d6add55 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.740115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.740850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.741125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.741395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.742058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:58.742353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:58.743747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:58.756209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 61/118] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:58.756924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.792543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aecd5f-0eb8-493b-b4d4-6d2b15bd029d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.804834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:58.831985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.832620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.832894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.833159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.833881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] RBAC: Adding query filter params (role.id=8183b92ce1a644b7b561153ffa9f474d, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=2a44996d1c774bbbab4566578c2fe8c2) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:58.834185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:58.835618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:58.840886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 58/119] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/role_assignments?user.id=2a44996d1c774bbbab4566578c2fe8c2&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=8183b92ce1a644b7b561153ffa9f474d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:58.841659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.857710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:58.874576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41b1b2-0410-442d-a602-f50d9b07b05a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.921453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.922904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:58.923357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:58.923755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:58.925521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:58.928205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:58.930455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 62/120] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:58.931368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:58.960109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a954b-8b17-498d-8e45-d8ea1ada12d5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:58.998612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52fdef-5d99-4e10-8d44-940da74d6797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:58.999681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52fdef-5d99-4e10-8d44-940da74d6797 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.000063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52fdef-5d99-4e10-8d44-940da74d6797 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.001967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52fdef-5d99-4e10-8d44-940da74d6797 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.014826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52fdef-5d99-4e10-8d44-940da74d6797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-8a52fdef-5d99-4e10-8d44-940da74d6797 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:36:59.020083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:36:59.021159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:36:59.022102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 59/121] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:36:59.023127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3604ba-4639-4e5c-98b8-b22c4aee958b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.023416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3604ba-4639-4e5c-98b8-b22c4aee958b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.023699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3604ba-4639-4e5c-98b8-b22c4aee958b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.025396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 60/122] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:36:59.026206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3604ba-4639-4e5c-98b8-b22c4aee958b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:59.065127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f409a-d550-4446-82f4-e5d3c3f30fbf None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:59.080377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:59.104312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:59.104963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.105499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.105778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.106932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:59.108455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:59.110818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 63/123] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:58 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:59.112070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:59.113380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:59.161780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3604ba-4639-4e5c-98b8-b22c4aee958b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:59.207618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c562e08-46c4-4dde-904d-5bacb40da561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:59.209013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c562e08-46c4-4dde-904d-5bacb40da561 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.209458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c562e08-46c4-4dde-904d-5bacb40da561 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.209889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c562e08-46c4-4dde-904d-5bacb40da561 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.211737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c562e08-46c4-4dde-904d-5bacb40da561 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:59.213643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c562e08-46c4-4dde-904d-5bacb40da561 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:59.216374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 61/124] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:59 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:59.217651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eb259-f98b-4e2f-93d4-3954dd8c0e09 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.218081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eb259-f98b-4e2f-93d4-3954dd8c0e09 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.218499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eb259-f98b-4e2f-93d4-3954dd8c0e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.229273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb9f56-13c9-4fb0-906b-aca7d5ac69bf None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:59.244662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-824eb259-f98b-4e2f-93d4-3954dd8c0e09 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:36:59.245913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-824eb259-f98b-4e2f-93d4-3954dd8c0e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:36:59.279064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:59.280014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.280532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.281005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.281881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:36:59.282632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:59.285067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:59.299222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 64/125] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:59.300344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:59.322786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 62/126] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:36:59.324415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824eb259-f98b-4e2f-93d4-3954dd8c0e09 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:59.350371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:59.406491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:59.445441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824eb259-f98b-4e2f-93d4-3954dd8c0e09 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:59.455401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff234816-85c5-460b-a4dc-a01ec06e9edc None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:59.485762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:59.486900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.487330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.487730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/a4740f00cc764a3ca9358d80c6fb5630/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.489823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9d09bb1714d98af3bd512e75cfe30, user_id=a4740f00cc764a3ca9358d80c6fb5630, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:59.492036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:59.516985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:59.517746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.518184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.518519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.519783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:59.520308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 63/127] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:36:59 2022] PUT /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/a4740f00cc764a3ca9358d80c6fb5630/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:36:59.521318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:59.521462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:59.530659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 65/128] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:59 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:59.537053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:59.726507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84113fd-56a3-4cf7-b8cb-1180992ccc8b None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:59.730710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f013a-b8a7-4df0-84db-536dc48c587d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:36:59.756873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:59.778279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:36:59.796023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:59.796700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.796982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.797390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] PATH_INFO: `/v3/users/28302e0508fb4cc195a19e42f7da3046` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.798661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:36:59.799329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:36:59.799604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:36:59.799876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/a4740f00cc764a3ca9358d80c6fb5630/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:36:59.801041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] RBAC: Authorizing `identity:get_user(user_id=28302e0508fb4cc195a19e42f7da3046)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:59.802526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:59.804303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 64/129] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:36:59 2022] GET /identity/v3/users/28302e0508fb4cc195a19e42f7da3046 => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:36:59.804983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:36:59.809053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9d09bb1714d98af3bd512e75cfe30, user_id=a4740f00cc764a3ca9358d80c6fb5630, role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:36:59.810542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:36:59.832897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 66/130] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:36:59 2022] PUT /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/a4740f00cc764a3ca9358d80c6fb5630/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:36:59.836545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:00.011415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f650e8-a708-4d4f-9e50-ff99c74a3b29 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:00.059561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483458a-f66a-474a-be55-50d7b7567856 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:00.086164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:00.087146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.087609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.088172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.105547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:00.106757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:00.107388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.107666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.107937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.109100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:00.110620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:00.112334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 67/131] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:36:59 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:00.112881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:00.113726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:00.114484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:00.120356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:00.121262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 65/132] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:36:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:00.127547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:00.167732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:00.212636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:00.236464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6071d6e-3814-432f-8e3b-b22181ea87f2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:00.266406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377caaa-efaa-4385-9981-e99d8f2c6072 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:00.273065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c138f-5131-48a4-9f27-c6a4be75d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:00.273746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c138f-5131-48a4-9f27-c6a4be75d3ea None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.274542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c138f-5131-48a4-9f27-c6a4be75d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.274827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c138f-5131-48a4-9f27-c6a4be75d3ea None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/28302e0508fb4cc195a19e42f7da3046/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.277279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c138f-5131-48a4-9f27-c6a4be75d3ea None admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9d09bb1714d98af3bd512e75cfe30, user_id=28302e0508fb4cc195a19e42f7da3046, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:00.279427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c138f-5131-48a4-9f27-c6a4be75d3ea None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:00.299194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 68/133] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:00 2022] PUT /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/28302e0508fb4cc195a19e42f7da3046/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:37:00.300586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2075a-5d4a-4e9b-af23-d05f020dcf3f None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.300874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2075a-5d4a-4e9b-af23-d05f020dcf3f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.301154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2075a-5d4a-4e9b-af23-d05f020dcf3f None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.302584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 69/134] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:00.308403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a437a7-c36a-4a24-b520-72c0c1d8eabe None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.308685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a437a7-c36a-4a24-b520-72c0c1d8eabe None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.308963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a437a7-c36a-4a24-b520-72c0c1d8eabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.309522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:00.310154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.310438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.310709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.311457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:00.311863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:00.313682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:00.332019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-96a437a7-c36a-4a24-b520-72c0c1d8eabe None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:00.333403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-96a437a7-c36a-4a24-b520-72c0c1d8eabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:00.337251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 66/135] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:00.347081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:00.426429 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 70/136] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:00.489569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295cb13-839c-4cf8-82f4-0d280eea0ed0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:00.514994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:00.541724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:00.542741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.543258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.543738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.545239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:00.546982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:00.549125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 67/137] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:00 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:00.554698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a437a7-c36a-4a24-b520-72c0c1d8eabe None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:00.593993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:00.679069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a437a7-c36a-4a24-b520-72c0c1d8eabe None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:00.714698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:00.715629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.716050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.716611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] PATH_INFO: `/v3/projects/33ae5df6b7d644ed85ddd9b056650998` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.717940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] RBAC: Authorizing `identity:get_project(project_id=33ae5df6b7d644ed85ddd9b056650998)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:00.719700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:00.721758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 71/138] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:00 2022] GET /identity/v3/projects/33ae5df6b7d644ed85ddd9b056650998 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:00.727635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:00.870538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a1039-2982-4a7b-b827-717811fbfb4a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:00.888157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:00.951567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153ed33-58b4-45b3-bcdc-8e52efbfdadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:00.952590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153ed33-58b4-45b3-bcdc-8e52efbfdadb None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:00.957049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153ed33-58b4-45b3-bcdc-8e52efbfdadb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:00.958688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153ed33-58b4-45b3-bcdc-8e52efbfdadb None admin] PATH_INFO: `/v3/projects/33ae5df6b7d644ed85ddd9b056650998/users/a4740f00cc764a3ca9358d80c6fb5630/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:00.962356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153ed33-58b4-45b3-bcdc-8e52efbfdadb None admin] RBAC: Authorizing `identity:create_grant(project_id=33ae5df6b7d644ed85ddd9b056650998, user_id=a4740f00cc764a3ca9358d80c6fb5630, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:00.964278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153ed33-58b4-45b3-bcdc-8e52efbfdadb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:00.989401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 68/139] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:00 2022] PUT /identity/v3/projects/33ae5df6b7d644ed85ddd9b056650998/users/a4740f00cc764a3ca9358d80c6fb5630/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:01.056500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83694c-d634-4a1a-9a4e-4fc930b23ad2 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:01.057083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83694c-d634-4a1a-9a4e-4fc930b23ad2 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:01.057533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83694c-d634-4a1a-9a4e-4fc930b23ad2 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:01.059049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 69/140] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:01.065646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0b3a5-4a91-463f-bfa4-8ea06efca671 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:01.066196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0b3a5-4a91-463f-bfa4-8ea06efca671 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:01.066656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0b3a5-4a91-463f-bfa4-8ea06efca671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:01.087520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:01.095282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89a484-f36e-48d2-9974-85f88a5e0a1d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:01.095463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ede0b3a5-4a91-463f-bfa4-8ea06efca671 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:01.099076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ede0b3a5-4a91-463f-bfa4-8ea06efca671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:01.143588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:01.144746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:01.146478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:01.147144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:01.149268 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:01.153590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:01.156016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 72/141] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:01.163526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:01.192663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:01.242529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 70/142] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:01.311967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe90a4d-bd2e-489c-898b-43417e33ce30 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:01.368907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:01.369824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:01.370288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:01.370742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:01.371565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:01.373262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:01.420320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 73/143] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:01 2022] POST /identity/v3/users => generated 283 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:01.590175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0b3a5-4a91-463f-bfa4-8ea06efca671 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:01.719775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0b3a5-4a91-463f-bfa4-8ea06efca671 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:01.769190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:01.770152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:01.770611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:01.771064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:01.789184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:01.794222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:01.795083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:01.795879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:01.797980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 71/144] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:01.803693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:01.837403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:01.931731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fdaf6-9a16-47ac-94f5-6a821a2d3fbb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:01.969016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:01.969929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:01.970354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:01.970785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:01.971659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:01.972093 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:01.973674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:01.984710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 74/145] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:01.992232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:02.029832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:02.108923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfaf8a-2129-4fb4-8bfd-568efdcd4622 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:02.149391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:02.150025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:02.150305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:02.150579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] PATH_INFO: `/v3/users/2a44996d1c774bbbab4566578c2fe8c2` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:02.151719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] RBAC: Authorizing `identity:get_user(user_id=2a44996d1c774bbbab4566578c2fe8c2)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:02.153391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:02.154972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 72/146] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:01 2022] GET /identity/v3/users/2a44996d1c774bbbab4566578c2fe8c2 => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:02.160873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:02.259642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:02.277517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2814-485a-495e-ae88-d1fec23e43c1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:02.319671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1e46d-2cc4-4b51-aeb7-068701cf9600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:02.320324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1e46d-2cc4-4b51-aeb7-068701cf9600 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:02.320603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1e46d-2cc4-4b51-aeb7-068701cf9600 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:02.320873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1e46d-2cc4-4b51-aeb7-068701cf9600 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:02.322025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1e46d-2cc4-4b51-aeb7-068701cf9600 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:02.323501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1e46d-2cc4-4b51-aeb7-068701cf9600 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:02.325502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 75/147] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:02 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:02.336237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:02.463550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daa6e3-a04d-4823-8b73-42e9eba66e9e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:02.466411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:02.504384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061499b8-1a35-4456-99bf-efee768ee991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:02.505400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061499b8-1a35-4456-99bf-efee768ee991 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:02.505958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061499b8-1a35-4456-99bf-efee768ee991 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:02.506441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061499b8-1a35-4456-99bf-efee768ee991 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/2a44996d1c774bbbab4566578c2fe8c2/roles/8183b92ce1a644b7b561153ffa9f474d` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:02.508614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061499b8-1a35-4456-99bf-efee768ee991 None admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9d09bb1714d98af3bd512e75cfe30, user_id=2a44996d1c774bbbab4566578c2fe8c2, role_id=8183b92ce1a644b7b561153ffa9f474d)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:02.510301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061499b8-1a35-4456-99bf-efee768ee991 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:02.527426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 73/148] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:02 2022] PUT /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/users/2a44996d1c774bbbab4566578c2fe8c2/roles/8183b92ce1a644b7b561153ffa9f474d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:03.081568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6899c6-c908-46be-9a8d-888845f94eaa None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:03.082109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6899c6-c908-46be-9a8d-888845f94eaa None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:03.082109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6899c6-c908-46be-9a8d-888845f94eaa None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:03.083533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 76/149] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:03.085064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8923c6-06b4-4bd0-a318-7f65f3bc9caa None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:03.085446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8923c6-06b4-4bd0-a318-7f65f3bc9caa None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:03.085604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8923c6-06b4-4bd0-a318-7f65f3bc9caa None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:03.087053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 74/150] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:03.093707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db8f22-1797-42df-970e-19b2b911435e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:03.093707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db8f22-1797-42df-970e-19b2b911435e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:03.093707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db8f22-1797-42df-970e-19b2b911435e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:03.099416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4f58-ab7e-49b5-9b89-925744f17432 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:03.101431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4f58-ab7e-49b5-9b89-925744f17432 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:03.102718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc4f58-ab7e-49b5-9b89-925744f17432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:03.119875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-09db8f22-1797-42df-970e-19b2b911435e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:03.126619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-09db8f22-1797-42df-970e-19b2b911435e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:03.142060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0dcc4f58-ab7e-49b5-9b89-925744f17432 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:03.143601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0dcc4f58-ab7e-49b5-9b89-925744f17432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:03.251943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 77/151] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:03.255020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1af854-1f75-47db-8b72-58abf0b3afda None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:03.255667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1af854-1f75-47db-8b72-58abf0b3afda None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:03.257772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1af854-1f75-47db-8b72-58abf0b3afda None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:03.259779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 78/152] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:03.261377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 75/153] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:03.265428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c238-9139-4f5f-97b8-9647a34b2387 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:03.266165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c238-9139-4f5f-97b8-9647a34b2387 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:03.266647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c238-9139-4f5f-97b8-9647a34b2387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:03.285321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ad57c238-9139-4f5f-97b8-9647a34b2387 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:03.286838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ad57c238-9139-4f5f-97b8-9647a34b2387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:03.357460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 79/154] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:03.677914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc4f58-ab7e-49b5-9b89-925744f17432 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:03.793865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c238-9139-4f5f-97b8-9647a34b2387 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:03.814126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:03.817910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:03.828926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc4f58-ab7e-49b5-9b89-925744f17432 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:03.874404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:03.875146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:03.875431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:03.875703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:03.894072 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:03.899147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:03.900083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:03.900881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:03.904858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 76/155] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:03.905716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.022418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c238-9139-4f5f-97b8-9647a34b2387 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.068194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed102a-8c98-4cd7-83d1-ff827d603887 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.088062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.088867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.089139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.089405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.113478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:04.119081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:04.119922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:04.120885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 80/156] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:04.121615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.141903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.142584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.142827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.143098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.168511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:04.173701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:04.175732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 77/157] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:04.177688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.205528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:04.250279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:04.290977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310bd82-1e48-45d8-ac48-48c96e530640 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.325049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.325655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.326085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.326251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.326841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:04.327137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:04.328600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:04.339418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 81/158] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:04.339418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.377287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db05a0-8bc1-4e51-b03e-23015de82e59 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.446675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7927cad-1fcc-471a-a223-0e21b5031736 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.454071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.454688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.454964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.455233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.455958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:04.456284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:04.457759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:04.474455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 78/159] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:04.475182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.485160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cc8f7-323b-4c4a-9f0c-c5e1444a9d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.485862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cc8f7-323b-4c4a-9f0c-c5e1444a9d55 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.486139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cc8f7-323b-4c4a-9f0c-c5e1444a9d55 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.486409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cc8f7-323b-4c4a-9f0c-c5e1444a9d55 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.487082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cc8f7-323b-4c4a-9f0c-c5e1444a9d55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:04.487382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cc8f7-323b-4c4a-9f0c-c5e1444a9d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:04.488807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cc8f7-323b-4c4a-9f0c-c5e1444a9d55 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:04.497580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 82/160] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:04.498691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5ccfe-3bab-4bc4-b136-c9516a04e70a None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.498982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5ccfe-3bab-4bc4-b136-c9516a04e70a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.499258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5ccfe-3bab-4bc4-b136-c9516a04e70a None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.500570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 83/161] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:04.501457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5ccfe-3bab-4bc4-b136-c9516a04e70a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.502117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:04.619492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:04.628303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5ccfe-3bab-4bc4-b136-c9516a04e70a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.662384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b150db3-1f63-40b8-8de4-88ed1a056206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.663017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150db3-1f63-40b8-8de4-88ed1a056206 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.663287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150db3-1f63-40b8-8de4-88ed1a056206 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.663554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150db3-1f63-40b8-8de4-88ed1a056206 None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.664699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b150db3-1f63-40b8-8de4-88ed1a056206 None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:04.666174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b150db3-1f63-40b8-8de4-88ed1a056206 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:04.667180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176308-dec6-408d-9214-d6aef17fa68e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.668057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 84/162] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:04 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:04.673688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cc4c9-2806-4bf8-be8b-0a5f31b2adf1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.674008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cc4c9-2806-4bf8-be8b-0a5f31b2adf1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.674254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cc4c9-2806-4bf8-be8b-0a5f31b2adf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.692557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2f1cc4c9-2806-4bf8-be8b-0a5f31b2adf1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:04.693742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2f1cc4c9-2806-4bf8-be8b-0a5f31b2adf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:04.739551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.740709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.741182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.741703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:04.744999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:04.746676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:04.748391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 79/163] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:04 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:04.753997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.762596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:04.765113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 85/164] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:04.772586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cc4c9-2806-4bf8-be8b-0a5f31b2adf1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:04.936918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cc4c9-2806-4bf8-be8b-0a5f31b2adf1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:04.980095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:04.981120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:04.981576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:04.982029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:05.032816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c004d5-6ef6-4e28-a42a-d4615a3daee4 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:05.053630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:05.061769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:05.063544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:05.106000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:05.106653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:05.106935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:05.107213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] PATH_INFO: `/v3/users/28302e0508fb4cc195a19e42f7da3046` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:05.109012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] RBAC: Authorizing `identity:get_user(user_id=28302e0508fb4cc195a19e42f7da3046)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:05.110710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:05.116396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 80/165] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:04 2022] GET /identity/v3/users/28302e0508fb4cc195a19e42f7da3046 => generated 247 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:05.117325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:05.142861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:05.183378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 86/166] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:05.184978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:05.307084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15955a-e31b-4fc2-8826-4a3f71f71126 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:05.312288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:05.318083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ced4f4-e5cd-44d0-b175-ff333ec78cfb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:05.378146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:05.379106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:05.379519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:05.379922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:05.394034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:05.395328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:05.395328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:05.395653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:05.469338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:05.471197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:05.524743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:05.526574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:05.552497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:05.580301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:05.589081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 87/167] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:05.590130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:05.707678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c5453-cdaa-4204-82f2-c317b6229464 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:05.720750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 81/168] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:05.725225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:05.745501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:05.746599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:05.747086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:05.747543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:05.749394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:05.751192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:05.753074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 88/169] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:05 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:05.754045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:05.838408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:05.865064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675062a-030f-42a3-a621-c55059c6d3b7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:05.902769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:05.903743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:05.904228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:05.904806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:05.906424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:05.908080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:05.909935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 89/170] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:05 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:05.910899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.011056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e39f3-4e38-4112-bf19-0c3408cbbbf1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:06.013425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1420f3-4aa8-42a5-a1d5-e400c91bdfee None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:06.027262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:06.048009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:06.051890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fab998-b70e-40a1-a211-cf6ea728bcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:06.052570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fab998-b70e-40a1-a211-cf6ea728bcc2 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.052843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fab998-b70e-40a1-a211-cf6ea728bcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.053270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fab998-b70e-40a1-a211-cf6ea728bcc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.055403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:06.056390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.056966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.057434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.058758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:06.060439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:06.062571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 90/171] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:05 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:06.063537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.065626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fab998-b70e-40a1-a211-cf6ea728bcc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-35fab998-b70e-40a1-a211-cf6ea728bcc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:06.070569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:06.071472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:06.072281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 82/172] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:06.072942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77faa-9769-4850-9df0-6fd850243247 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.073686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77faa-9769-4850-9df0-6fd850243247 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.073686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77faa-9769-4850-9df0-6fd850243247 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.074834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 83/173] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:06.075539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd77faa-9769-4850-9df0-6fd850243247 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.173446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81de3e-a645-493f-925d-b8c6e64fc2ab None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:06.185520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd77faa-9769-4850-9df0-6fd850243247 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:06.210048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629a51c-66c3-4c7e-aa78-6a7eac121597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:06.211330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a51c-66c3-4c7e-aa78-6a7eac121597 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.211802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a51c-66c3-4c7e-aa78-6a7eac121597 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.212372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a51c-66c3-4c7e-aa78-6a7eac121597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.213354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a51c-66c3-4c7e-aa78-6a7eac121597 None admin] RBAC: Adding query filter params (role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=a4740f00cc764a3ca9358d80c6fb5630) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:06.213866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a51c-66c3-4c7e-aa78-6a7eac121597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:06.215570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a51c-66c3-4c7e-aa78-6a7eac121597 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:06.221920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:06.222536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.222799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.223067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.223811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 91/174] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:06 2022] GET /identity/v3/role_assignments?user.id=a4740f00cc764a3ca9358d80c6fb5630&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:06.224089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=28302e0508fb4cc195a19e42f7da3046) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:06.224429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:06.225734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dc850-17b0-418f-8c37-9f98077042c9 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.225904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:06.226383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dc850-17b0-418f-8c37-9f98077042c9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.226839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dc850-17b0-418f-8c37-9f98077042c9 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.228754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 92/175] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:06.229742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dc850-17b0-418f-8c37-9f98077042c9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.237314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 84/176] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:06 2022] GET /identity/v3/role_assignments?user.id=28302e0508fb4cc195a19e42f7da3046&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:06.238628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.278697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:06.306363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:06.352719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dc850-17b0-418f-8c37-9f98077042c9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:06.387926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fce2f3-fd7f-44f6-8ae6-fd825b0d4d46 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:06.395705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd1cbd-aa50-48d8-bb6d-f175ffb7d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:06.396701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1cbd-aa50-48d8-bb6d-f175ffb7d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.397247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1cbd-aa50-48d8-bb6d-f175ffb7d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.397749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1cbd-aa50-48d8-bb6d-f175ffb7d6c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.398689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd1cbd-aa50-48d8-bb6d-f175ffb7d6c4 None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=a4740f00cc764a3ca9358d80c6fb5630) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:06.399211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd1cbd-aa50-48d8-bb6d-f175ffb7d6c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:06.401029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd1cbd-aa50-48d8-bb6d-f175ffb7d6c4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:06.408569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 93/177] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:06 2022] GET /identity/v3/role_assignments?user.id=a4740f00cc764a3ca9358d80c6fb5630&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:06.410192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4481d-5210-42e9-8e56-1df5d92fb817 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.410763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4481d-5210-42e9-8e56-1df5d92fb817 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.411225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4481d-5210-42e9-8e56-1df5d92fb817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.431507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-40c4481d-5210-42e9-8e56-1df5d92fb817 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:06.433179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-40c4481d-5210-42e9-8e56-1df5d92fb817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:06.516182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 94/178] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:06.518302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9716553-b70b-411a-9da1-74e45c2811fc None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.518935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9716553-b70b-411a-9da1-74e45c2811fc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.519490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9716553-b70b-411a-9da1-74e45c2811fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.541697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9143661-dfa0-45c7-b99b-6824de558ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:06.542326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9143661-dfa0-45c7-b99b-6824de558ed4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.542606 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9143661-dfa0-45c7-b99b-6824de558ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.542888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9143661-dfa0-45c7-b99b-6824de558ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.543559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9143661-dfa0-45c7-b99b-6824de558ed4 None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:06.543860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9143661-dfa0-45c7-b99b-6824de558ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:06.545325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9143661-dfa0-45c7-b99b-6824de558ed4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:06.546059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a9716553-b70b-411a-9da1-74e45c2811fc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:06.548055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a9716553-b70b-411a-9da1-74e45c2811fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:06.566626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 85/179] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:06.568041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a85d-d02b-4dad-9046-319fc967a38c None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.568353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a85d-d02b-4dad-9046-319fc967a38c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.568628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a85d-d02b-4dad-9046-319fc967a38c None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.569903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 86/180] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:06.575669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a85d-d02b-4dad-9046-319fc967a38c None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.639521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:06.670623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 95/181] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:06.672276 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47863-46f0-471a-bcbe-90d36ecba7c5 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.672876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47863-46f0-471a-bcbe-90d36ecba7c5 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.673968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47863-46f0-471a-bcbe-90d36ecba7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.705352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-04c47863-46f0-471a-bcbe-90d36ecba7c5 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:06.706470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-04c47863-46f0-471a-bcbe-90d36ecba7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:06.753858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:06.759284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a85d-d02b-4dad-9046-319fc967a38c None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:06.809370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:06.810360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:06.810864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:06.811320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] PATH_INFO: `/v3/users/a4740f00cc764a3ca9358d80c6fb5630` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:06.813094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] RBAC: Authorizing `identity:get_user(user_id=a4740f00cc764a3ca9358d80c6fb5630)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:06.816803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:06.818706 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 87/182] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:06 2022] GET /identity/v3/users/a4740f00cc764a3ca9358d80c6fb5630 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:06.825954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.872485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 96/183] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:06.901560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c47863-46f0-471a-bcbe-90d36ecba7c5 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:06.963985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d79472-6108-42ce-9ac5-d9f94cc47b47 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.011411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.012022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.012311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.012581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.023842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c47863-46f0-471a-bcbe-90d36ecba7c5 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.041317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.057103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.059830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652ee33-7512-457e-a76f-fda2c6a2b467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.060543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ee33-7512-457e-a76f-fda2c6a2b467 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.060799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ee33-7512-457e-a76f-fda2c6a2b467 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.061068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ee33-7512-457e-a76f-fda2c6a2b467 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.071600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652ee33-7512-457e-a76f-fda2c6a2b467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a652ee33-7512-457e-a76f-fda2c6a2b467 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:07.076445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:07.077317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 97/184] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:07.083624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38829d-da78-4783-99d5-b620a3d1412b None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.084372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38829d-da78-4783-99d5-b620a3d1412b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.084372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38829d-da78-4783-99d5-b620a3d1412b None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.084372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 98/185] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:07.084372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38829d-da78-4783-99d5-b620a3d1412b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.099988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.101572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.184101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38829d-da78-4783-99d5-b620a3d1412b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.205036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 88/186] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:07.205036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.220880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853c86c-4bfe-4213-800d-c61774df9cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.221434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853c86c-4bfe-4213-800d-c61774df9cda None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.221703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853c86c-4bfe-4213-800d-c61774df9cda None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.221967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853c86c-4bfe-4213-800d-c61774df9cda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.231765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853c86c-4bfe-4213-800d-c61774df9cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-8853c86c-4bfe-4213-800d-c61774df9cda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:07.236583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:07.237448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 99/187] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:07.240953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b1005-167a-4df5-9c68-d7a4353da13f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.241679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b1005-167a-4df5-9c68-d7a4353da13f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.241679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b1005-167a-4df5-9c68-d7a4353da13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.257707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1f7b1005-167a-4df5-9c68-d7a4353da13f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:07.258897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1f7b1005-167a-4df5-9c68-d7a4353da13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:07.273006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.289626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.312788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f2744-8d99-4654-844c-df4d736f7276 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.331238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 100/188] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:07.332136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b1005-167a-4df5-9c68-d7a4353da13f None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.347521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.348513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.348959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.349396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.350744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:07.351042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.352488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.361146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 89/189] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.368322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.439598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b1005-167a-4df5-9c68-d7a4353da13f None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.473442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.473985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.474149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f8e22-1756-473c-b783-de00e6311098 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.474244 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.474505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.485591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:07.490367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:07.491236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:07.492013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 101/190] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:07.493189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.509796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.518290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.518919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.519191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.519558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] PATH_INFO: `/v3/projects/33ae5df6b7d644ed85ddd9b056650998` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.520778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] RBAC: Authorizing `identity:get_project(project_id=33ae5df6b7d644ed85ddd9b056650998)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.522464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.522817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.525096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 90/191] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/projects/33ae5df6b7d644ed85ddd9b056650998 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.525705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.604187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb653b6-d203-4f04-a060-5353b4c8c72a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.629819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53264c-64bb-4fb0-a18e-bfaa362c32f1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.637073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.637678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.637945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.638215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.639189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:07.639479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.640937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.648653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 102/192] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.649420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.668492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.669119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.669395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.669667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] PATH_INFO: `/v3/users/c68bf7cd699e486f88abe6f3e58e524e` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.683172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] RBAC: Authorizing `identity:get_user(user_id=c68bf7cd699e486f88abe6f3e58e524e)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.684849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.686449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 91/193] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/users/c68bf7cd699e486f88abe6f3e58e524e => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.691288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.734974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.742876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.748160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41222acd-8596-4f59-bbd1-cb61b6f080e1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.781486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.782075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.782340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.782621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.783277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:07.783565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.784965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.788236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d91fc0-bf95-4d63-81b3-8e6455fb5a88 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.792764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 103/194] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.793333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.823604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.824205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.824507 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.824773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.825494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=33ae5df6b7d644ed85ddd9b056650998, user.id=a4740f00cc764a3ca9358d80c6fb5630) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:07.825797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.827241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.832952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 92/195] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/role_assignments?user.id=a4740f00cc764a3ca9358d80c6fb5630&scope.project.id=33ae5df6b7d644ed85ddd9b056650998&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.833720 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.901205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fac2e-b231-41e1-bc27-189e7f029e98 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.937970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.938688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.938963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.939262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.940468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.941967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.943600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 104/196] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.945055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951d76e-06e4-42fe-b880-26e26b221f32 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:07.945258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:07.965781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.972045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:07.988794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:07.989317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:07.989609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:07.989914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] PATH_INFO: `/v3/users/2a44996d1c774bbbab4566578c2fe8c2` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:07.991068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=2a44996d1c774bbbab4566578c2fe8c2)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:07.992589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:07.999047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 93/197] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/users/2a44996d1c774bbbab4566578c2fe8c2 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:07.999998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.068940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cba08-94da-49a8-9927-b922167f9301 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.100685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.101280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.101554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.101815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.104864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47b9d-432f-4e07-849c-72937ce6a8a6 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.137579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.138197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.138466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.139058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] PATH_INFO: `/v3/users/c68bf7cd699e486f88abe6f3e58e524e` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.140198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] RBAC: Authorizing `identity:get_user(user_id=c68bf7cd699e486f88abe6f3e58e524e)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.141813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.143383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 94/198] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/users/c68bf7cd699e486f88abe6f3e58e524e => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:08.144107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.177704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.179181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.196486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.201114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.244830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8332c5-6079-4230-b2ee-808985377c53 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.275202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 105/199] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:08.276046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.279049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.280043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.280043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.280284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.281229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.282821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.303636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 95/200] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:08 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:08.304397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.394128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddae46-c2a5-41fe-b099-42066e7194e2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.414025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.429591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628d7d0-54e7-4584-ad57-720b1e6dad60 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.433955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.434577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.434852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.435119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.442952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.473966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.473966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.473966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.473966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.523824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.525271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.563403 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.564965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.646741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 106/201] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:08.647853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.670218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.695493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 96/202] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:08.696458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.714997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.761989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab0ac6-0074-4c14-913e-4436c512e490 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.811024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.812091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.813466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.813754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.815190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.817366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.819656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 107/203] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:08 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:08.820660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.830839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7363e1-411d-4ed4-a5f6-4fde1154cfcd None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.867218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.867955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.868312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.868652 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.869969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.871594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.873345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 97/204] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:08 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:08.874252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:08.932150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.948819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce327b06-19cb-409a-af82-bfee101d5ed1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.968725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:08.988036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:08.988927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:08.989264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:08.989593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:08.991004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:08.992600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:08.993732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcfae9-39a7-46d3-9362-67285c9fb250 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:08.994710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 108/205] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:08 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:08.995711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:09.045466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:09.046706 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:09.047599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:09.048030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:09.049266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=c68bf7cd699e486f88abe6f3e58e524e) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:09.049579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:09.051707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:09.059117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 98/206] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:08 2022] GET /identity/v3/role_assignments?user.id=c68bf7cd699e486f88abe6f3e58e524e&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:09.060014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:09.183361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee200af-bf85-406f-8fed-e023eb753134 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:09.226485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e3342-4234-48c2-8ce0-320be4015ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:09.227361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e3342-4234-48c2-8ce0-320be4015ef4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:09.227636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e3342-4234-48c2-8ce0-320be4015ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:09.227906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e3342-4234-48c2-8ce0-320be4015ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:09.228815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e3342-4234-48c2-8ce0-320be4015ef4 None admin] RBAC: Adding query filter params (role.id=8183b92ce1a644b7b561153ffa9f474d, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30, user.id=2a44996d1c774bbbab4566578c2fe8c2) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:09.229390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e3342-4234-48c2-8ce0-320be4015ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:09.231767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e3342-4234-48c2-8ce0-320be4015ef4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:09.240052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 109/207] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:08 2022] GET /identity/v3/role_assignments?user.id=2a44996d1c774bbbab4566578c2fe8c2&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=8183b92ce1a644b7b561153ffa9f474d => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:09.260979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482dbe-9aec-48a3-a19c-ab8d41401d38 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:09.303582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9b8e4-f2c9-4d19-84aa-acb53a2934c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:09.304205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b8e4-f2c9-4d19-84aa-acb53a2934c1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:09.304515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b8e4-f2c9-4d19-84aa-acb53a2934c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:09.304785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b8e4-f2c9-4d19-84aa-acb53a2934c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:09.305593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9b8e4-f2c9-4d19-84aa-acb53a2934c1 None admin] RBAC: Adding query filter params (role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=c68bf7cd699e486f88abe6f3e58e524e) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:09.305904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9b8e4-f2c9-4d19-84aa-acb53a2934c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:09.307434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9b8e4-f2c9-4d19-84aa-acb53a2934c1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:09.314097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 99/208] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:09 2022] GET /identity/v3/role_assignments?user.id=c68bf7cd699e486f88abe6f3e58e524e&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:10.399250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58953d6-28fe-4431-9081-d755dcdde87f None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:10.399924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58953d6-28fe-4431-9081-d755dcdde87f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:10.399924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58953d6-28fe-4431-9081-d755dcdde87f None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:10.401145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 110/209] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:10.407175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc13fab-80ed-401a-ae7e-f21c1c8b20be None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:10.407609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc13fab-80ed-401a-ae7e-f21c1c8b20be None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:10.407778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc13fab-80ed-401a-ae7e-f21c1c8b20be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:10.421530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:10.426948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-abc13fab-80ed-401a-ae7e-f21c1c8b20be None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:10.428329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-abc13fab-80ed-401a-ae7e-f21c1c8b20be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:10.436715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9345ec3-abfe-4841-8c7e-c341c31eed45 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:10.437300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9345ec3-abfe-4841-8c7e-c341c31eed45 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:10.437780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9345ec3-abfe-4841-8c7e-c341c31eed45 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:10.439322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 111/210] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:10.444494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6111b-1dbc-4871-be4c-23b86e370114 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:10.444974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6111b-1dbc-4871-be4c-23b86e370114 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:10.445412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6111b-1dbc-4871-be4c-23b86e370114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:10.467273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-08e6111b-1dbc-4871-be4c-23b86e370114 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:10.468840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-08e6111b-1dbc-4871-be4c-23b86e370114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:10.486371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:10.501581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 100/211] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:10.549537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 112/212] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:10.782098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc13fab-80ed-401a-ae7e-f21c1c8b20be None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:10.822920 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6111b-1dbc-4871-be4c-23b86e370114 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:10.891988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc13fab-80ed-401a-ae7e-f21c1c8b20be None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:10.926973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:10.927846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:10.928311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:10.928732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:10.931134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6111b-1dbc-4871-be4c-23b86e370114 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:10.931735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:10.940546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:10.945463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:10.946537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:10.947525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:10.949361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 101/213] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:10.955913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:10.976627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:10.977766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:10.978211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:10.978620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:10.980317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:10.981913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:10.984304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 113/214] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:10.996287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:11.020404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:11.088176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e94675-67c6-4195-87a3-1c868ce546c7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:11.103249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9ce9b-146d-478d-b268-6739dfb51c6a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:11.125424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:11.126357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.126795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.127215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.128060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:11.128595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:11.130245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:11.139836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560e0b-b063-4aad-8526-ad416a9da1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:11.140619 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 102/215] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:11.145358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560e0b-b063-4aad-8526-ad416a9da1d4 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.145635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560e0b-b063-4aad-8526-ad416a9da1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.145904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560e0b-b063-4aad-8526-ad416a9da1d4 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.146561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560e0b-b063-4aad-8526-ad416a9da1d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:11.148038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560e0b-b063-4aad-8526-ad416a9da1d4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:11.149205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:11.176423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 114/216] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:10 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:11.181415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecd4b-bdd2-4991-895e-b6d51f6ce9bd None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.181767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecd4b-bdd2-4991-895e-b6d51f6ce9bd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.182108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecd4b-bdd2-4991-895e-b6d51f6ce9bd None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.183636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 115/217] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:11.189587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb118f-8366-4705-8b93-879ded9410fc None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.190089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb118f-8366-4705-8b93-879ded9410fc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.190577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb118f-8366-4705-8b93-879ded9410fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.209962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-97eb118f-8366-4705-8b93-879ded9410fc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:11.211569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-97eb118f-8366-4705-8b93-879ded9410fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:11.228553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:11.293083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a78d7-8608-4c4d-8ebb-a67142c6003b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:11.304741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 116/218] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:11.331497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1312876-cef4-4a98-a8ed-a00e8e8cd1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:11.332139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1312876-cef4-4a98-a8ed-a00e8e8cd1d8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.332448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1312876-cef4-4a98-a8ed-a00e8e8cd1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.332713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1312876-cef4-4a98-a8ed-a00e8e8cd1d8 None admin] PATH_INFO: `/v3/users/9d5421defff84ad59f89fe8fa81036e6` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.352337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1312876-cef4-4a98-a8ed-a00e8e8cd1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=9d5421defff84ad59f89fe8fa81036e6)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:11.353914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1312876-cef4-4a98-a8ed-a00e8e8cd1d8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:11.360529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 103/219] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:11 2022] GET /identity/v3/users/9d5421defff84ad59f89fe8fa81036e6 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:11.362181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb118f-8366-4705-8b93-879ded9410fc None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:11.363602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292af14c-b595-465d-be58-8d349a270b1f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.363904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292af14c-b595-465d-be58-8d349a270b1f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.364137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292af14c-b595-465d-be58-8d349a270b1f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.368303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:11.368477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 104/220] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:11.371448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056066d6-06f4-4152-89d9-66ddb54bdc39 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.371831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056066d6-06f4-4152-89d9-66ddb54bdc39 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.372179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056066d6-06f4-4152-89d9-66ddb54bdc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.391950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-056066d6-06f4-4152-89d9-66ddb54bdc39 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:11.393155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-056066d6-06f4-4152-89d9-66ddb54bdc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:11.475787 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 105/221] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:11.480012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb118f-8366-4705-8b93-879ded9410fc None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:11.521664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:11.522606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.523081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.523528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.578975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056066d6-06f4-4152-89d9-66ddb54bdc39 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:11.596334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:11.598459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:11.600465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:11.642120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:11.711919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056066d6-06f4-4152-89d9-66ddb54bdc39 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:11.724592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 117/222] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:11.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:11.759678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:11.760385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.760661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.760925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.773779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:11.778485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:11.779373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 106/223] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:11.781147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:11.847916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fbdf7-7156-4c5a-b837-5248b9601eae None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:11.851308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:11.886481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:11.887355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.887635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.887905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.889919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:11.891591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:11.892391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557db0a4-8885-412f-a9e1-95a435e9b911 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:11.898018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 118/224] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:11 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:11.898018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:11.905265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:11.933242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:11.934218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:11.934676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:11.935154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:11.946136 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:11.951275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:11.952147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:11.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:11.955695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 107/225] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:11.956905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.017249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef6007-dcbf-4b1b-9f4d-1354c4f362c1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.054380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9011-a567-492d-a740-289d07111351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.055009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9011-a567-492d-a740-289d07111351 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.055296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9011-a567-492d-a740-289d07111351 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.055527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9011-a567-492d-a740-289d07111351 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.056194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f9011-a567-492d-a740-289d07111351 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:12.056539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f9011-a567-492d-a740-289d07111351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.058016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f9011-a567-492d-a740-289d07111351 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.061210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6da1cd-ad6a-4ff3-982a-a16dbcb8f6b8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.067101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 119/226] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:12.067699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9011-a567-492d-a740-289d07111351 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.098105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.099067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.099527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.100007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.100882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:12.101175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=9d5421defff84ad59f89fe8fa81036e6) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:12.101708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.103461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.110027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 108/227] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:11 2022] GET /identity/v3/role_assignments?user.id=9d5421defff84ad59f89fe8fa81036e6&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:12.111235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.144218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:12.181669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9011-a567-492d-a740-289d07111351 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.219242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.219861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.220137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.220428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.221105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:12.221405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.221913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199183-9790-41b8-b237-4bc088f185c8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.222830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.232384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 120/228] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:12.238391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.260668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.261714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.262231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.262682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] PATH_INFO: `/v3/users/c68bf7cd699e486f88abe6f3e58e524e` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.264363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] RBAC: Authorizing `identity:get_user(user_id=c68bf7cd699e486f88abe6f3e58e524e)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.266305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.268413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 109/229] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:12 2022] GET /identity/v3/users/c68bf7cd699e486f88abe6f3e58e524e => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:12.276437 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.393189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:12.401532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab9838-c860-475a-ab45-1f6e1d563f95 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.438846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:12.443292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea49ae-b9f3-492e-b1b6-6822df8daf3a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.479690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.480698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.482500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.482959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] PATH_INFO: `/v3/users/c68bf7cd699e486f88abe6f3e58e524e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.485375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] RBAC: Authorizing `identity:get_user(user_id=c68bf7cd699e486f88abe6f3e58e524e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.488432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.494165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 121/230] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:12 2022] GET /identity/v3/users/c68bf7cd699e486f88abe6f3e58e524e => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:12.501490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.501952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.502603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.502877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.503150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.504368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.505860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.511929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 110/231] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:12 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:12.513096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.658292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838beee8-0521-4b35-be49-64ca1ccda0b6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.693838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.694703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.694977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.695243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.700091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.700091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.700091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 122/232] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:12 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:12.706540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:12.734885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e7f3e-0c23-4af7-a9cf-c583c58b1380 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.748286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:12.784420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79476e-9c4c-4dea-a737-bec82b18e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.785446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79476e-9c4c-4dea-a737-bec82b18e214 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.785803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79476e-9c4c-4dea-a737-bec82b18e214 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.786459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79476e-9c4c-4dea-a737-bec82b18e214 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/c68bf7cd699e486f88abe6f3e58e524e/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.788487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79476e-9c4c-4dea-a737-bec82b18e214 None admin] RBAC: Authorizing `identity:create_grant(project_id=1840a5c96dff47be87e068418ad64f3c, user_id=c68bf7cd699e486f88abe6f3e58e524e, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.790780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79476e-9c4c-4dea-a737-bec82b18e214 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.808524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 111/233] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:12 2022] PUT /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/c68bf7cd699e486f88abe6f3e58e524e/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:37:12.884708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0c4bc-4d24-4a49-a288-0495b3ffa04a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:12.943939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739fc49-b770-4dae-a277-b93dc0d014d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:12.944561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fc49-b770-4dae-a277-b93dc0d014d5 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:12.944839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fc49-b770-4dae-a277-b93dc0d014d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:12.945113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fc49-b770-4dae-a277-b93dc0d014d5 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/c68bf7cd699e486f88abe6f3e58e524e/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:12.947407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739fc49-b770-4dae-a277-b93dc0d014d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1840a5c96dff47be87e068418ad64f3c, user_id=c68bf7cd699e486f88abe6f3e58e524e, role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:12.948908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739fc49-b770-4dae-a277-b93dc0d014d5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:12.966559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 123/234] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:12 2022] PUT /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/c68bf7cd699e486f88abe6f3e58e524e/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:13.629805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e66fe7-34d3-472b-905d-49dfb2c53b99 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:13.630345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e66fe7-34d3-472b-905d-49dfb2c53b99 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:13.630345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e66fe7-34d3-472b-905d-49dfb2c53b99 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:13.631868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 112/235] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:13.637259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9055c-06f7-4dd1-bedd-29fd436bea22 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:13.637649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9055c-06f7-4dd1-bedd-29fd436bea22 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:13.637829 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9055c-06f7-4dd1-bedd-29fd436bea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:13.656362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-88d9055c-06f7-4dd1-bedd-29fd436bea22 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:13.657601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-88d9055c-06f7-4dd1-bedd-29fd436bea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:13.726748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c091ff6-1cd8-482e-823f-e43fdab05a7f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:13.727054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c091ff6-1cd8-482e-823f-e43fdab05a7f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:13.727284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c091ff6-1cd8-482e-823f-e43fdab05a7f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:13.728891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 113/236] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:13.737096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f0dd5-369f-4aa1-abbe-acaa3e1e44fa None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:13.737662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f0dd5-369f-4aa1-abbe-acaa3e1e44fa None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:13.738127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f0dd5-369f-4aa1-abbe-acaa3e1e44fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:13.758913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 124/237] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:13.759700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3d3f0dd5-369f-4aa1-abbe-acaa3e1e44fa None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:13.761190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3d3f0dd5-369f-4aa1-abbe-acaa3e1e44fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:13.783389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:13.835327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 114/238] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:14.134036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9055c-06f7-4dd1-bedd-29fd436bea22 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:14.161680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:14.167410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f0dd5-369f-4aa1-abbe-acaa3e1e44fa None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:14.302438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9055c-06f7-4dd1-bedd-29fd436bea22 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:14.317467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f0dd5-369f-4aa1-abbe-acaa3e1e44fa None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:14.357037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:14.357909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:14.358294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:14.358590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:14.359950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:14.361538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:14.363328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 125/239] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:14.368889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:14.378966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:14.380248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:14.380989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:14.381644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:14.403665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:14.416791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:14.417898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:14.418870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:14.421176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 115/240] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:14.428408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:14.467776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:14.468667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:14.481471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa98c31-c9d5-4539-a098-f49b36552c15 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:14.518577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:14.519348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:14.519717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:14.520081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:14.521030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:14.522593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:14.546303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 126/241] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:14 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:14.669454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e66787-ebd6-4b27-be37-c6cec62b6d5f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:14.720435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64db4-f09a-4019-b37d-7003a156320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:14.721049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64db4-f09a-4019-b37d-7003a156320d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:14.721324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64db4-f09a-4019-b37d-7003a156320d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:14.721588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64db4-f09a-4019-b37d-7003a156320d None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:14.722262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64db4-f09a-4019-b37d-7003a156320d None admin] RBAC: Adding query filter params (name=reader) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:14.722557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64db4-f09a-4019-b37d-7003a156320d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:14.723936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64db4-f09a-4019-b37d-7003a156320d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:14.734470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 116/242] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:14.740675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:14.818616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36fcb1-94f4-4618-8c72-ff26fbdd8f66 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:14.818921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36fcb1-94f4-4618-8c72-ff26fbdd8f66 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:14.819160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36fcb1-94f4-4618-8c72-ff26fbdd8f66 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:14.821736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 117/243] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:14.825693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb6633-faf0-4b47-98ed-381ae2cae742 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:14.825967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb6633-faf0-4b47-98ed-381ae2cae742 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:14.826230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb6633-faf0-4b47-98ed-381ae2cae742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:14.844755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-30bb6633-faf0-4b47-98ed-381ae2cae742 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:14.845862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-30bb6633-faf0-4b47-98ed-381ae2cae742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:14.855262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db62d8-0564-467b-ba6d-87e04467c9d1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:14.905571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:14.906428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:14.906703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:14.906969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] PATH_INFO: `/v3/users/ca3966fe94f4454fa99650823c101d7e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:14.924057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] RBAC: Authorizing `identity:get_user(user_id=ca3966fe94f4454fa99650823c101d7e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:14.925507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:14.927197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 127/244] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:14 2022] GET /identity/v3/users/ca3966fe94f4454fa99650823c101d7e => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:14.932618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 118/245] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:14.933220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:14.937309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.049572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34feab-5058-4ca6-b221-1ba478e19c56 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:15.086528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:15.087469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.087960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.088424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.102533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2014-d3f9-43b9-9029-54592c378372 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.102710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2014-d3f9-43b9-9029-54592c378372 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.102968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2014-d3f9-43b9-9029-54592c378372 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.104581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 119/246] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:15.110348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e926ffb-2800-42f7-a37f-8402a8a2ddea None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.110620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e926ffb-2800-42f7-a37f-8402a8a2ddea None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.110883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e926ffb-2800-42f7-a37f-8402a8a2ddea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.129964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8e926ffb-2800-42f7-a37f-8402a8a2ddea None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:15.130489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8e926ffb-2800-42f7-a37f-8402a8a2ddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:15.132655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.171663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:15.171663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:15.172836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.234535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 120/247] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:15.245909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e926ffb-2800-42f7-a37f-8402a8a2ddea None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:15.283919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 128/248] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:15.290774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:15.390327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e926ffb-2800-42f7-a37f-8402a8a2ddea None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:15.415541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676f4ec-a3bc-43c4-8121-f57a3155e4fe None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:15.417042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.428075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22922c9d-90c7-481f-9b1e-4cc13c9da645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:15.429076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922c9d-90c7-481f-9b1e-4cc13c9da645 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.429565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922c9d-90c7-481f-9b1e-4cc13c9da645 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.430077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922c9d-90c7-481f-9b1e-4cc13c9da645 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.442639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22922c9d-90c7-481f-9b1e-4cc13c9da645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-22922c9d-90c7-481f-9b1e-4cc13c9da645 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:15.448015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:15.448928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:15.449785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:15.450828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3837fbe-870e-4cba-ae49-09ad3a5de845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:15.451290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837fbe-870e-4cba-ae49-09ad3a5de845 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.451564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837fbe-870e-4cba-ae49-09ad3a5de845 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.451832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837fbe-870e-4cba-ae49-09ad3a5de845 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.456803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3837fbe-870e-4cba-ae49-09ad3a5de845 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:15.457136 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.457238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3837fbe-870e-4cba-ae49-09ad3a5de845 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:15.457238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 129/250] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:15 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:15.457495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 121/250] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:15.458311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c90d1-804e-4761-abed-eadba389bc59 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.458582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c90d1-804e-4761-abed-eadba389bc59 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.458846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c90d1-804e-4761-abed-eadba389bc59 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.460036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 130/251] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:15.465731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c90d1-804e-4761-abed-eadba389bc59 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:15.467053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbf947-1449-4581-bbb3-f68d0f77241a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.467332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbf947-1449-4581-bbb3-f68d0f77241a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.467599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbf947-1449-4581-bbb3-f68d0f77241a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.491462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a7fbf947-1449-4581-bbb3-f68d0f77241a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:15.493029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a7fbf947-1449-4581-bbb3-f68d0f77241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:15.577244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 122/252] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:15.578187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbf947-1449-4581-bbb3-f68d0f77241a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:15.590775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c90d1-804e-4761-abed-eadba389bc59 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:15.631290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:15.632231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.632733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.633569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.634647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] RBAC: Adding query filter params (role.id=8183b92ce1a644b7b561153ffa9f474d, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=ca3966fe94f4454fa99650823c101d7e) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:15.635137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:15.637168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:15.648600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 131/253] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:15 2022] GET /identity/v3/role_assignments?user.id=ca3966fe94f4454fa99650823c101d7e&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=8183b92ce1a644b7b561153ffa9f474d => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:15.650843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:15.700998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.776947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70c15e-ed6c-49bd-8066-ac1dc9b4883f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:15.815838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:15.816516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.816796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.817070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.829038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:15.835297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:15.836275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:15.837658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 132/254] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:15.842918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:15.900825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbf947-1449-4581-bbb3-f68d0f77241a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:15.946570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.953790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7afa0-df1c-4c5e-a1cd-24dc4c0b8de0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:15.979525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:15.994301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:15.994917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.995193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.995300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:15.995494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.995877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:15.996126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:15.996221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:15.996455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:15.996561 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:15.997180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:15.997480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:15.997836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:15.998846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:16.008903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 133/255] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:16.009217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 123/256] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:16.010629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.015055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.118844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64e1a4-0d4a-40ff-a6a0-fd594e09f624 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:16.149264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b92ee-67a9-4268-9a0a-8e9bb51ad8fc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:16.163488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:16.165161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:16.165643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:16.166349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:16.181217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:16.185594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:16.186223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:16.186498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:16.186805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] PATH_INFO: `/v3/users/c68bf7cd699e486f88abe6f3e58e524e` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:16.187989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=c68bf7cd699e486f88abe6f3e58e524e)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:16.188371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:16.189234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:16.190044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:16.190640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:16.191163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 124/257] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/users/c68bf7cd699e486f88abe6f3e58e524e => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:16.191887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.198565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 134/258] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:16.198565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.222106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:16.257359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:16.325181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402534e1-2fe2-4261-b602-2ceeb0255ed7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:16.364689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:16.365324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:16.365596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:16.365865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] PATH_INFO: `/v3/users/9d5421defff84ad59f89fe8fa81036e6` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:16.370059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] RBAC: Authorizing `identity:get_user(user_id=9d5421defff84ad59f89fe8fa81036e6)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:16.371482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:16.376702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 125/259] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/users/9d5421defff84ad59f89fe8fa81036e6 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:16.378439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.388796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c402c3e-8597-44c7-9804-13e94d209c12 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:16.485508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:16.486122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:16.486389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:16.486650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:16.496745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be83c-d756-4e9c-9309-e596d221a687 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:16.524362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:16.537168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:16.538726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:16.539750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:16.540333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:16.542069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:16.542687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:16.545659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:16.564213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 126/260] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:16.565506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.612741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:16.614585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:16.634130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:16.741376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315d8e5-9c90-4efe-ace7-182cbb706993 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:16.787737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:16.788284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 135/261] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:16.789152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.792426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:16.792716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:16.792992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:16.794182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:16.800622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:16.802476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 127/262] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:16.803371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:16.917164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:16.968673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133450ec-00c6-48c2-806e-1de66cad0bfe None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:16.986402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb28238-c2b6-4e93-a763-7903cbaf0bbd None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:17.002753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:17.003201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.003477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.003748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.005148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:17.010073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:17.010073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 128/263] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:17.010073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:17.030026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:17.099263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:17.100386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.100852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.101303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] PATH_INFO: `/v3/users/c68bf7cd699e486f88abe6f3e58e524e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.104913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] RBAC: Authorizing `identity:get_user(user_id=c68bf7cd699e486f88abe6f3e58e524e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:17.106645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:17.108555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 136/264] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:16 2022] GET /identity/v3/users/c68bf7cd699e486f88abe6f3e58e524e => generated 283 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:17.109582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:17.119499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94246936-e7f0-4154-9203-e0a69c05dc84 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:17.153602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:17.162487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:17.163441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.164556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.165182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/9d5421defff84ad59f89fe8fa81036e6/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.169638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] RBAC: Authorizing `identity:create_grant(project_id=1840a5c96dff47be87e068418ad64f3c, user_id=9d5421defff84ad59f89fe8fa81036e6, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:17.171625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:17.211279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 129/265] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:17 2022] PUT /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/9d5421defff84ad59f89fe8fa81036e6/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:17.213108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:17.294550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930dc42-cccb-417f-9168-e4622a11a558 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:17.377756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb55586-5b08-45a5-a778-a41a6a57733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:17.380858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb55586-5b08-45a5-a778-a41a6a57733b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.381390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb55586-5b08-45a5-a778-a41a6a57733b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.382102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb55586-5b08-45a5-a778-a41a6a57733b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.384566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb55586-5b08-45a5-a778-a41a6a57733b None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=c68bf7cd699e486f88abe6f3e58e524e) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:17.385139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb55586-5b08-45a5-a778-a41a6a57733b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:17.390754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb55586-5b08-45a5-a778-a41a6a57733b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:17.403509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 137/266] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:17 2022] GET /identity/v3/role_assignments?user.id=c68bf7cd699e486f88abe6f3e58e524e&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:17.405926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a3e3-c375-484d-9af0-5310e88c3a47 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.407771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a3e3-c375-484d-9af0-5310e88c3a47 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.409737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a3e3-c375-484d-9af0-5310e88c3a47 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 138/267] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:17.417713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a594fe-9bd6-470c-adf8-96f5d6a986cd None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.418266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a594fe-9bd6-470c-adf8-96f5d6a986cd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.418753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a594fe-9bd6-470c-adf8-96f5d6a986cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.454708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5102-3225-4d60-b834-ec7c35e47dfc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:17.457546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-00a594fe-9bd6-470c-adf8-96f5d6a986cd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:17.459535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-00a594fe-9bd6-470c-adf8-96f5d6a986cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:17.496328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2fdca-5533-4b15-abd8-b77f5fcc8406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:17.497309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fdca-5533-4b15-abd8-b77f5fcc8406 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.497804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fdca-5533-4b15-abd8-b77f5fcc8406 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.498264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2fdca-5533-4b15-abd8-b77f5fcc8406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.562812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:17.568746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:17.583550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2fdca-5533-4b15-abd8-b77f5fcc8406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:17.585569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2fdca-5533-4b15-abd8-b77f5fcc8406 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:17.623054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 139/268] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:17.714205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 130/269] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:17.720157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a594fe-9bd6-470c-adf8-96f5d6a986cd None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:17.856450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a594fe-9bd6-470c-adf8-96f5d6a986cd None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:17.896400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:17.897026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.897291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.897552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.898652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:17.900151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:17.903153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 140/270] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:17 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:17.905966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef02a8-affc-4214-91ff-9282d499e4bc None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.906241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef02a8-affc-4214-91ff-9282d499e4bc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.906510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef02a8-affc-4214-91ff-9282d499e4bc None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.907968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 131/271] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:17.908785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:17.920448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493cae-5252-4105-ac0c-221be9eddefe None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:17.920964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493cae-5252-4105-ac0c-221be9eddefe None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:17.921052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493cae-5252-4105-ac0c-221be9eddefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:17.945337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e4493cae-5252-4105-ac0c-221be9eddefe None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:17.946524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e4493cae-5252-4105-ac0c-221be9eddefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:17.984475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:18.039022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 132/272] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:18.040227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4493cae-5252-4105-ac0c-221be9eddefe None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:18.047120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eeb6f-0f29-465a-9790-13456b433c98 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.058387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:18.141573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f4a1c-5b8e-414d-8ef4-1fcdf4f2fe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.142176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f4a1c-5b8e-414d-8ef4-1fcdf4f2fe87 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.142439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f4a1c-5b8e-414d-8ef4-1fcdf4f2fe87 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.142702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f4a1c-5b8e-414d-8ef4-1fcdf4f2fe87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.143437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f4a1c-5b8e-414d-8ef4-1fcdf4f2fe87 None admin] RBAC: Adding query filter params (role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=c68bf7cd699e486f88abe6f3e58e524e) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:18.143732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f4a1c-5b8e-414d-8ef4-1fcdf4f2fe87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:18.145203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f4a1c-5b8e-414d-8ef4-1fcdf4f2fe87 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:18.159248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 141/273] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:17 2022] GET /identity/v3/role_assignments?user.id=c68bf7cd699e486f88abe6f3e58e524e&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:18.165276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4493cae-5252-4105-ac0c-221be9eddefe None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.225381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.226420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.226880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.227327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.237689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4b266-5741-466d-a1a6-cd30c6015c99 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.237856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4b266-5741-466d-a1a6-cd30c6015c99 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.238095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4b266-5741-466d-a1a6-cd30c6015c99 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.239765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 142/274] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:18.245024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1e26-ac7c-47f4-b588-f8a3fd6fb65d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.245300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1e26-ac7c-47f4-b588-f8a3fd6fb65d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.245570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1e26-ac7c-47f4-b588-f8a3fd6fb65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.251264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:18.257516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:18.258397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:18.259212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:18.261478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 133/275] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:18.267775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:18.280164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-c16e1e26-ac7c-47f4-b588-f8a3fd6fb65d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:18.280164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c16e1e26-ac7c-47f4-b588-f8a3fd6fb65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:18.394465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 143/276] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:18.398980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e1e26-ac7c-47f4-b588-f8a3fd6fb65d None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:18.399926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:18.435938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce820ea5-541e-4694-b6be-9191ca1f1864 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.456206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:18.479973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.481092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.481844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.482556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.483547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:18.484037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:18.485794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:18.496822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 134/277] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:18.502767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:18.532867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e1e26-ac7c-47f4-b588-f8a3fd6fb65d None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.570600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.571777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.572241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.572727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.574162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:18.576086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:18.578221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 144/278] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:18.585707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:18.626260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb46a7-49cc-4336-a721-bbde1fda185c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.663696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.664372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.664656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.665022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] PATH_INFO: `/v3/users/de25dde2b70c427daf4de38cd5670f84` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.668715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:18.683190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] RBAC: Authorizing `identity:get_user(user_id=de25dde2b70c427daf4de38cd5670f84)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:18.684717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:18.686404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 135/279] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:18 2022] GET /identity/v3/users/de25dde2b70c427daf4de38cd5670f84 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:18.692619 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:18.720872 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fce6e2-f82b-4eb7-9319-cce1f996fa9b None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.732165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:18.760222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.760812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.761090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.761365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.762010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:18.763818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:18.786529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 145/280] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:18 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:18.787491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:18.810551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1661a-253b-48cc-8e9a-df99d319c42f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.852061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.852949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.853401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.853851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.854676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, user.id=de25dde2b70c427daf4de38cd5670f84, scope.system=all) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:18.855048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:18.856927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:18.863964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 136/281] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Aug 12 10:37:18 2022] GET /identity/v3/role_assignments?user.id=de25dde2b70c427daf4de38cd5670f84&scope.system=all&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:18.918017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc316c9-cff1-4bdf-85bf-28268a239306 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:18.960565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:18.961209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:18.961483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:18.961750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:18.974888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:18.980338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:18.981261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:18.982037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 146/282] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:18.989413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:19.063919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:19.112243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00795098-8bb5-476c-9654-4cfd3d05c45c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:19.150491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:19.151098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:19.151371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:19.151636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:19.152324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] RBAC: Adding query filter params (name=reader) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:19.152622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:19.154008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:19.163229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 137/283] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:19.168739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:19.202371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:19.285047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d303-f4b0-4f36-90e3-37f21bdd8723 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:19.320698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:19.321679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:19.322145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:19.322604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] PATH_INFO: `/v3/users/ca3966fe94f4454fa99650823c101d7e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:19.323923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=ca3966fe94f4454fa99650823c101d7e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:19.325628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:19.328438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 147/284] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:19 2022] GET /identity/v3/users/ca3966fe94f4454fa99650823c101d7e => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:19.337883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:19.491060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3b725-75f2-4f78-afca-e215bf8b2e6f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:19.518817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:19.539667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1753b-245a-4150-914e-724c31cdeb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:19.540318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1753b-245a-4150-914e-724c31cdeb05 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:19.540597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1753b-245a-4150-914e-724c31cdeb05 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:19.540868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1753b-245a-4150-914e-724c31cdeb05 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:19.542000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1753b-245a-4150-914e-724c31cdeb05 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:19.543446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1753b-245a-4150-914e-724c31cdeb05 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:19.545423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 138/285] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:19 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:19.551076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:19.603086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903a5f-1879-4b6a-9bdb-5c425fa5d31c None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:19.603442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903a5f-1879-4b6a-9bdb-5c425fa5d31c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:19.603909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903a5f-1879-4b6a-9bdb-5c425fa5d31c None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:19.606719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 139/286] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:19.612393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa551b-efe1-4f01-9e43-bfe03c99f4ac None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:19.612669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa551b-efe1-4f01-9e43-bfe03c99f4ac None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:19.612937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa551b-efe1-4f01-9e43-bfe03c99f4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:19.640959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-82aa551b-efe1-4f01-9e43-bfe03c99f4ac None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:19.642167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-82aa551b-efe1-4f01-9e43-bfe03c99f4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:19.690794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:19.699514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3d616-32dd-4143-8913-81cd62f11cd2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:19.735972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:19.736673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:19.736945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:19.737212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/ca3966fe94f4454fa99650823c101d7e/roles/8183b92ce1a644b7b561153ffa9f474d` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:19.739088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1840a5c96dff47be87e068418ad64f3c, user_id=ca3966fe94f4454fa99650823c101d7e, role_id=8183b92ce1a644b7b561153ffa9f474d)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:19.740568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:19.758538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 148/287] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:19 2022] PUT /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c/users/ca3966fe94f4454fa99650823c101d7e/roles/8183b92ce1a644b7b561153ffa9f474d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:19.783639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 140/288] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:20.083971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:20.216173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2163b2-12eb-4d85-a35b-170224e100c1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:20.259312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:20.259940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:20.260209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:20.260497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:20.279710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:20.285147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:20.286019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:20.287225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 149/289] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:20.294040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa551b-efe1-4f01-9e43-bfe03c99f4ac None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:20.459630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa551b-efe1-4f01-9e43-bfe03c99f4ac None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:20.481595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:20.503691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:20.504314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:20.504589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:20.504855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:20.505510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:20.505805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:20.507226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:20.516239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 141/290] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:20.522059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:20.557155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:20.670927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e93d9-8b46-4816-a6d5-c14c5fcb16e4 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:20.714165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:20.714839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:20.715111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:20.715392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] PATH_INFO: `/v3/users/9d5421defff84ad59f89fe8fa81036e6` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:20.718296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] RBAC: Authorizing `identity:get_user(user_id=9d5421defff84ad59f89fe8fa81036e6)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:20.719807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:20.721460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 150/291] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:20 2022] GET /identity/v3/users/9d5421defff84ad59f89fe8fa81036e6 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:20.727362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:20.872050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cacb0e-3681-4e5f-bb35-551181971982 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:20.921150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4641d2-e347-49f0-874b-b27c54ebd76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:20.921698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4641d2-e347-49f0-874b-b27c54ebd76f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:20.921973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4641d2-e347-49f0-874b-b27c54ebd76f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:20.922238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4641d2-e347-49f0-874b-b27c54ebd76f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:20.987572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:21.012098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4641d2-e347-49f0-874b-b27c54ebd76f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:21.013927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4641d2-e347-49f0-874b-b27c54ebd76f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:21.134016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 142/292] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:21.139809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:21.268581 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:21.291225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf8ba1-5883-49d6-a08f-b703b42749c3 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:21.315448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fabc80-ffa5-412f-84b8-ed5cf72c7215 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.315965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fabc80-ffa5-412f-84b8-ed5cf72c7215 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.315965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fabc80-ffa5-412f-84b8-ed5cf72c7215 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.317648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 143/293] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:21.323432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703682-c6cd-46ee-8792-bbb43043dac4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.323757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703682-c6cd-46ee-8792-bbb43043dac4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.323969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703682-c6cd-46ee-8792-bbb43043dac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.349532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:21.350177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.350452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.350724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.351854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:21.355444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:21.355444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 151/294] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:21 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:21.356633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-65703682-c6cd-46ee-8792-bbb43043dac4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:21.358062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-65703682-c6cd-46ee-8792-bbb43043dac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:21.375992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:21.437279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 144/295] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:21.438602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5311b-93d4-4a0c-a04f-800fb769824f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.438892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5311b-93d4-4a0c-a04f-800fb769824f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.439165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5311b-93d4-4a0c-a04f-800fb769824f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.440574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 145/296] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:21.445578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4aa83-0e81-4ff9-8a1e-9472b39632a1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.445858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4aa83-0e81-4ff9-8a1e-9472b39632a1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.446128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4aa83-0e81-4ff9-8a1e-9472b39632a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.463943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f5e4aa83-0e81-4ff9-8a1e-9472b39632a1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:21.470035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f5e4aa83-0e81-4ff9-8a1e-9472b39632a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:21.470035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:21.497144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b32228-29ec-4713-ba9e-c66becb6f756 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:21.532622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:21.533596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.534032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.534469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.534680 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 146/297] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:21.535721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=9d5421defff84ad59f89fe8fa81036e6) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:21.536240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:21.537902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:21.552304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d2a0-6ad2-4fa2-850f-66c2739f1129 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.552304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d2a0-6ad2-4fa2-850f-66c2739f1129 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.552304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d2a0-6ad2-4fa2-850f-66c2739f1129 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.552304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 147/299] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:21.552966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 152/299] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:21 2022] GET /identity/v3/role_assignments?user.id=9d5421defff84ad59f89fe8fa81036e6&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:21.564826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3ae19-81e9-474d-a15b-5a853828dfab None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.565113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3ae19-81e9-474d-a15b-5a853828dfab None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.565504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3ae19-81e9-474d-a15b-5a853828dfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.600993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2fa3ae19-81e9-474d-a15b-5a853828dfab None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:21.602220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2fa3ae19-81e9-474d-a15b-5a853828dfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:21.693942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 148/300] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:21.707797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:21.738642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:21.827643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c17e0-ebff-4caa-9222-886f4d5d7ea8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:21.864336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:21.864969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:21.865240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:21.865502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:21.866615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:21.868114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:21.869955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 153/301] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:21.875617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3ae19-81e9-474d-a15b-5a853828dfab None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:21.944246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:21.954324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:21.981408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3ae19-81e9-474d-a15b-5a853828dfab None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.033304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.033914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.034182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.034443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] PATH_INFO: `/v3/groups` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.035060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:22.036580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:22.053604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:22.056223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 149/302] 10.222.0.23 () {60 vars in 1134 bytes} [Fri Aug 12 10:37:21 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:37:22.057217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.074900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366b47d-09d5-459f-9767-856d6a47ed20 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.112140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.112838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.113112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.113457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.125611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:22.132484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:22.133444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:22.134241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 154/303] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:22.141108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.168746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6dd6e-542a-4bff-975a-afa365d9cdc2 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.200249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:22.207070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.208528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.208809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.209183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.221462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:22.227549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:22.229460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:22.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 150/304] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:22.235034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.278166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5575a-5d7b-4b64-a2af-dd2596d76819 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.316782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f500145-90e5-42be-b1e8-59b6d4f23693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.317850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f500145-90e5-42be-b1e8-59b6d4f23693 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.318126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f500145-90e5-42be-b1e8-59b6d4f23693 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.318394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f500145-90e5-42be-b1e8-59b6d4f23693 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.319387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f500145-90e5-42be-b1e8-59b6d4f23693 None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:22.319836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f500145-90e5-42be-b1e8-59b6d4f23693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:22.321297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f500145-90e5-42be-b1e8-59b6d4f23693 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:22.330000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 155/305] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:22.331040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fd1d4-d019-4f55-8e76-4c13f61a30d9 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.331321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fd1d4-d019-4f55-8e76-4c13f61a30d9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.331591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fd1d4-d019-4f55-8e76-4c13f61a30d9 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.336306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 156/306] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:22.337847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fd1d4-d019-4f55-8e76-4c13f61a30d9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.341553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:22.352148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a1824-87b0-4489-abee-3ba4639c41b5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.385446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056b6f1-825a-4296-a5de-272b83482189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.385964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056b6f1-825a-4296-a5de-272b83482189 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.386238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056b6f1-825a-4296-a5de-272b83482189 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.386499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056b6f1-825a-4296-a5de-272b83482189 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.387154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056b6f1-825a-4296-a5de-272b83482189 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:22.387446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056b6f1-825a-4296-a5de-272b83482189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:22.389182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056b6f1-825a-4296-a5de-272b83482189 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:22.397607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 151/307] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:22.398778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618327fa-91c4-490c-a180-eb6dfde44f16 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.399059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618327fa-91c4-490c-a180-eb6dfde44f16 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.399324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618327fa-91c4-490c-a180-eb6dfde44f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.417431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-618327fa-91c4-490c-a180-eb6dfde44f16 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:22.418593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-618327fa-91c4-490c-a180-eb6dfde44f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:22.456476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fd1d4-d019-4f55-8e76-4c13f61a30d9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.466331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:22.485944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 152/308] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:22.486791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618327fa-91c4-490c-a180-eb6dfde44f16 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.502798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.504059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.504801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.505476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] PATH_INFO: `/v3/users/de25dde2b70c427daf4de38cd5670f84` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.507328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=de25dde2b70c427daf4de38cd5670f84)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:22.509443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:22.516605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 157/309] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/users/de25dde2b70c427daf4de38cd5670f84 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:22.517579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.601225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618327fa-91c4-490c-a180-eb6dfde44f16 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.616684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:22.634277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffd980-a610-40f8-8965-5002370b5fd6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.646907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.647698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.647971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.648236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] PATH_INFO: `/v3/users/2eafc41f6c0043678e5d33e010eecc76` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.669779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] RBAC: Authorizing `identity:get_user(user_id=2eafc41f6c0043678e5d33e010eecc76)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:22.671611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:22.673492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 153/310] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/users/2eafc41f6c0043678e5d33e010eecc76 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:22.678397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.678397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.678397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.678397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] PATH_INFO: `/v3/system/users/de25dde2b70c427daf4de38cd5670f84/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.678672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=de25dde2b70c427daf4de38cd5670f84, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:22.678672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:22.680180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.689484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 158/311] 10.222.0.23 () {58 vars in 1334 bytes} [Fri Aug 12 10:37:22 2022] PUT /identity/v3/system/users/de25dde2b70c427daf4de38cd5670f84/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:22.732444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:22.778259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:22.809737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e228c9-e1aa-41bd-9941-cebd25c0fc96 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.860009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.861254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.861839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.862110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.862841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] RBAC: Adding query filter params (role.id=8183b92ce1a644b7b561153ffa9f474d, user.id=2eafc41f6c0043678e5d33e010eecc76, scope.system=all) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:22.863145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:22.864664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:22.872093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 154/312] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/role_assignments?user.id=2eafc41f6c0043678e5d33e010eecc76&scope.system=all&role.id=8183b92ce1a644b7b561153ffa9f474d => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:22.911301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738c22c-a4c8-4816-96b2-41041fcb2eee None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:22.950217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:22.951159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:22.951572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:22.952060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:22.954630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:22.966919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:22.973073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:22.974212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:22.975169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:22.977520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 159/313] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:22.986012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:23.094285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ef26-d92b-4154-b418-38605e9f0f43 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:23.096631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:23.129988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:23.130591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:23.130862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:23.131123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:23.131780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:23.132070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:23.133485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:23.142438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 155/314] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:23.148281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:23.252493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a6c98-38bf-41de-85ff-0f2c56be61c4 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:23.286388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:23.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:23.287347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:23.287620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] PATH_INFO: `/v3/users/ca3966fe94f4454fa99650823c101d7e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:23.288777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] RBAC: Authorizing `identity:get_user(user_id=ca3966fe94f4454fa99650823c101d7e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:23.290240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:23.292173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 160/315] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:23 2022] GET /identity/v3/users/ca3966fe94f4454fa99650823c101d7e => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:23.297861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:23.403187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346a037-c038-4ef1-ab84-1003ed536045 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:23.436599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:23.437361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:23.437637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:23.437904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:23.472788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:23.505654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:23.507024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:23.603671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 156/316] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:23.609409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:23.672684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:23.711682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ac8b7-a7dd-4bb1-82d4-35ed3c65263d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:23.746750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e49a2e-923d-4ebf-9722-ee084e596167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:23.747754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e49a2e-923d-4ebf-9722-ee084e596167 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:23.748242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e49a2e-923d-4ebf-9722-ee084e596167 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:23.748746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e49a2e-923d-4ebf-9722-ee084e596167 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:23.750091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e49a2e-923d-4ebf-9722-ee084e596167 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:23.751754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e49a2e-923d-4ebf-9722-ee084e596167 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:23.754122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 161/317] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:23 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:23.760247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:23.824444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:23.859126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba8c9b-ae0d-466c-a88c-f89c97c31041 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:23.892934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de65006-1790-4397-8649-00a95ca88b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:23.893623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de65006-1790-4397-8649-00a95ca88b9e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:23.893903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de65006-1790-4397-8649-00a95ca88b9e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:23.894217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de65006-1790-4397-8649-00a95ca88b9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:23.894964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de65006-1790-4397-8649-00a95ca88b9e None admin] RBAC: Adding query filter params (role.id=8183b92ce1a644b7b561153ffa9f474d, scope.project.id=1840a5c96dff47be87e068418ad64f3c, user.id=ca3966fe94f4454fa99650823c101d7e) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:23.895266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de65006-1790-4397-8649-00a95ca88b9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:23.896791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de65006-1790-4397-8649-00a95ca88b9e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:23.903755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 157/318] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:23 2022] GET /identity/v3/role_assignments?user.id=ca3966fe94f4454fa99650823c101d7e&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=8183b92ce1a644b7b561153ffa9f474d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:24.084242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3207f-c857-4529-aec8-bdb6f27193bb None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.084915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3207f-c857-4529-aec8-bdb6f27193bb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.085391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3207f-c857-4529-aec8-bdb6f27193bb None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.086939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 162/319] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:24.092373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acaebd-f55c-4233-9b79-a69cce572e20 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.092987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acaebd-f55c-4233-9b79-a69cce572e20 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.093627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acaebd-f55c-4233-9b79-a69cce572e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.112966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-98acaebd-f55c-4233-9b79-a69cce572e20 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:24.114464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-98acaebd-f55c-4233-9b79-a69cce572e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:24.182052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 158/320] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:24.444221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3207f-c857-4529-aec8-bdb6f27193bb None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:24.545423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3207f-c857-4529-aec8-bdb6f27193bb None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:24.574048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:24.581452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db784a-a882-48ee-ac3a-b1f2afd30376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:24.582082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db784a-a882-48ee-ac3a-b1f2afd30376 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.582354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db784a-a882-48ee-ac3a-b1f2afd30376 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.582621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db784a-a882-48ee-ac3a-b1f2afd30376 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.583725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db784a-a882-48ee-ac3a-b1f2afd30376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:24.585187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db784a-a882-48ee-ac3a-b1f2afd30376 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:24.586967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 163/321] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:24.592364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acaebd-f55c-4233-9b79-a69cce572e20 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:24.654352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:24.697407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acaebd-f55c-4233-9b79-a69cce572e20 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:24.731794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5c640-ae6f-4db7-b4f0-ee30591ab479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:24.732429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5c640-ae6f-4db7-b4f0-ee30591ab479 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.732700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5c640-ae6f-4db7-b4f0-ee30591ab479 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.732963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5c640-ae6f-4db7-b4f0-ee30591ab479 None admin] PATH_INFO: `/v3/groups` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.733753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5c640-ae6f-4db7-b4f0-ee30591ab479 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:24.735320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5c640-ae6f-4db7-b4f0-ee30591ab479 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:24.752177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 159/322] 10.222.0.23 () {60 vars in 1134 bytes} [Fri Aug 12 10:37:24 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:24.781201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee3ac8-f6b7-4ca3-894e-6d057c972525 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.781498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee3ac8-f6b7-4ca3-894e-6d057c972525 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.781939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee3ac8-f6b7-4ca3-894e-6d057c972525 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.783730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 164/323] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:24.789501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369765fc-a2d5-457b-8aa8-805e2bb5b1b4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.789728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369765fc-a2d5-457b-8aa8-805e2bb5b1b4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.790082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369765fc-a2d5-457b-8aa8-805e2bb5b1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.810553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-369765fc-a2d5-457b-8aa8-805e2bb5b1b4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:24.812182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-369765fc-a2d5-457b-8aa8-805e2bb5b1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:24.858397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c67107-6170-4f69-b5cc-40dafd56e4d5 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.858600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c67107-6170-4f69-b5cc-40dafd56e4d5 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.858878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c67107-6170-4f69-b5cc-40dafd56e4d5 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.860479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 165/324] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:24.866022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95b9bb-491d-4e43-9faa-394ded2be55d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:24.866332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95b9bb-491d-4e43-9faa-394ded2be55d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:24.866526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95b9bb-491d-4e43-9faa-394ded2be55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:24.885228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ac95b9bb-491d-4e43-9faa-394ded2be55d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:24.886447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ac95b9bb-491d-4e43-9faa-394ded2be55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:24.890915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 160/325] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:24.961043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 166/326] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:25.180126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369765fc-a2d5-457b-8aa8-805e2bb5b1b4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.235363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95b9bb-491d-4e43-9faa-394ded2be55d None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.237244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:25.293292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369765fc-a2d5-457b-8aa8-805e2bb5b1b4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.345905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95b9bb-491d-4e43-9faa-394ded2be55d None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.347590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.348640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.349110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.349565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.362441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:25.367674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:25.368569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:25.369369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:25.372020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 161/327] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:25.378845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.386395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:25.394289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.394930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.395203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.395469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.406125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:25.411162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:25.412068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:25.413538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 167/328] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:25.419234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.490467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92254faa-6a94-47c6-b8ef-eef617237512 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.501923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:25.528108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.528909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.529179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d8b45-2a2e-4b29-863d-0ef785b007a9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.529606 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.529725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.530545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:25.531024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:25.532552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:25.541790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 162/329] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:25.548399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.575439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.576336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.576776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.577210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.578222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:25.578714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:25.580903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:25.595763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 168/330] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:25.600911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.671158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b3b7-244b-43f2-a66a-e2bdcaf9b3b3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.692681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:25.708573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.709367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.709742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.710008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] PATH_INFO: `/v3/users/de25dde2b70c427daf4de38cd5670f84` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.711363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] RBAC: Authorizing `identity:get_user(user_id=de25dde2b70c427daf4de38cd5670f84)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:25.712856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:25.714440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 163/331] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:25 2022] GET /identity/v3/users/de25dde2b70c427daf4de38cd5670f84 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:25.720696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.722518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f55448-ebd3-4439-b914-5dd6745a7642 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.760665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.761217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.761490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.761759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] PATH_INFO: `/v3/users/2eafc41f6c0043678e5d33e010eecc76` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.763303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] RBAC: Authorizing `identity:get_user(user_id=2eafc41f6c0043678e5d33e010eecc76)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:25.764938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:25.766843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 169/332] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:25 2022] GET /identity/v3/users/2eafc41f6c0043678e5d33e010eecc76 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:25.769645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:25.772664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:25.831271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae954492-2ac6-4a4b-9da4-059f51853102 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.869847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f62bb-bdb7-4280-9811-db3ae2e9dced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.870453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f62bb-bdb7-4280-9811-db3ae2e9dced None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.870722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f62bb-bdb7-4280-9811-db3ae2e9dced None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.870986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f62bb-bdb7-4280-9811-db3ae2e9dced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.872091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f62bb-bdb7-4280-9811-db3ae2e9dced None admin] RBAC: Adding query filter params (role.id=b96d83bc5d78463dbf0415bc66a0b0db, user.id=de25dde2b70c427daf4de38cd5670f84, scope.system=all) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:25.872495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f62bb-bdb7-4280-9811-db3ae2e9dced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:25.875003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f62bb-bdb7-4280-9811-db3ae2e9dced None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:25.885388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63ec9d-29bb-42e2-bf1c-78ef8ff0c88f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:25.890891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 164/333] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Aug 12 10:37:25 2022] GET /identity/v3/role_assignments?user.id=de25dde2b70c427daf4de38cd5670f84&scope.system=all&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:25.936747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:25.970015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce96081-62c2-4a19-8b7b-15f69875b5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:25.970680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce96081-62c2-4a19-8b7b-15f69875b5d7 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:25.970952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce96081-62c2-4a19-8b7b-15f69875b5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:25.971219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce96081-62c2-4a19-8b7b-15f69875b5d7 None admin] PATH_INFO: `/v3/system/users/2eafc41f6c0043678e5d33e010eecc76/roles/8183b92ce1a644b7b561153ffa9f474d` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:25.972647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce96081-62c2-4a19-8b7b-15f69875b5d7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2eafc41f6c0043678e5d33e010eecc76, role_id=8183b92ce1a644b7b561153ffa9f474d)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:25.974033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce96081-62c2-4a19-8b7b-15f69875b5d7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:25.992615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 170/334] 10.222.0.23 () {58 vars in 1334 bytes} [Fri Aug 12 10:37:25 2022] PUT /identity/v3/system/users/2eafc41f6c0043678e5d33e010eecc76/roles/8183b92ce1a644b7b561153ffa9f474d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:37:27.085902 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3456be-bd0a-48d7-9b2d-2ec74e8f752d None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:27.086918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3456be-bd0a-48d7-9b2d-2ec74e8f752d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:27.087391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3456be-bd0a-48d7-9b2d-2ec74e8f752d None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:27.089264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 165/335] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:27.095988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b986420-dda4-4e43-94bc-b60b9d273c16 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:27.096853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b986420-dda4-4e43-94bc-b60b9d273c16 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:27.097336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b986420-dda4-4e43-94bc-b60b9d273c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:27.116145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1b986420-dda4-4e43-94bc-b60b9d273c16 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:27.117819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1b986420-dda4-4e43-94bc-b60b9d273c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:27.188083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 171/336] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:27.469335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3456be-bd0a-48d7-9b2d-2ec74e8f752d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:27.585329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:27.590177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3456be-bd0a-48d7-9b2d-2ec74e8f752d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:27.632826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b542a6b-5dd2-4eaa-856d-fdc22f3b3fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:27.633455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542a6b-5dd2-4eaa-856d-fdc22f3b3fb7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:27.633731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542a6b-5dd2-4eaa-856d-fdc22f3b3fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:27.633999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542a6b-5dd2-4eaa-856d-fdc22f3b3fb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:27.647357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b542a6b-5dd2-4eaa-856d-fdc22f3b3fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-4b542a6b-5dd2-4eaa-856d-fdc22f3b3fb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:27.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:27.653501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:27.654283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 166/337] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:27.660057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b986420-dda4-4e43-94bc-b60b9d273c16 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:27.753328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72651-5b7b-40e0-9c8a-cc36ae93fc38 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:27.753678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72651-5b7b-40e0-9c8a-cc36ae93fc38 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:27.753875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e72651-5b7b-40e0-9c8a-cc36ae93fc38 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:27.755216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 167/338] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:27.766011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979371-737e-472d-b3da-08edba27363d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:27.766581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979371-737e-472d-b3da-08edba27363d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:27.767053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979371-737e-472d-b3da-08edba27363d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:27.770526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b986420-dda4-4e43-94bc-b60b9d273c16 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:27.784962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:27.808130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-01979371-737e-472d-b3da-08edba27363d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:27.810529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-01979371-737e-472d-b3da-08edba27363d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:27.812204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:27.812858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:27.813137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:27.813404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:27.817177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:27.817177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:27.817177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:27.829193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 172/339] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:27.830422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:27.922944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 168/340] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:27.936319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966ff99-c8fb-4648-95f5-ff24bf00ee56 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:27.975795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:27.976439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:27.976714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:27.976996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.024626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:28.053406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:28.054961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:28.156760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 173/341] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:37:28.162551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979371-737e-472d-b3da-08edba27363d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:28.243524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:28.271113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:28.290650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979371-737e-472d-b3da-08edba27363d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:28.359791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:28.360790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.361285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.361754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.365059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:28.366719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:28.368621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 169/342] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:28 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:28.374579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:28.392614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:28.458231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ecd5-c139-4458-8f1e-9d363f3d86a2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:28.527911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d70a91-12bd-442b-ba33-a410dbfbd266 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:28.553017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:28.553840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.554110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.554390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.569093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd530a-4e3c-4ab9-a729-fcccd3e8fc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:28.570232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd530a-4e3c-4ab9-a729-fcccd3e8fc3a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.570651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd530a-4e3c-4ab9-a729-fcccd3e8fc3a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.571089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd530a-4e3c-4ab9-a729-fcccd3e8fc3a None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.579876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd530a-4e3c-4ab9-a729-fcccd3e8fc3a None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:28.581212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:28.582005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd530a-4e3c-4ab9-a729-fcccd3e8fc3a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:28.583840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 170/343] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:28 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:28.586196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:28.587258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:28.588158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:28.589842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32e968-1578-4c93-b3f6-1e7ffd67d69f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.590124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32e968-1578-4c93-b3f6-1e7ffd67d69f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.592549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32e968-1578-4c93-b3f6-1e7ffd67d69f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.592549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 171/344] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:28.593748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32e968-1578-4c93-b3f6-1e7ffd67d69f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:28.595135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 174/345] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:28.595878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:28.620234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:28.718427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32e968-1578-4c93-b3f6-1e7ffd67d69f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:28.765173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb43d2-c1bc-41ad-9101-e6463c9ada36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:28.766313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb43d2-c1bc-41ad-9101-e6463c9ada36 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.766765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb43d2-c1bc-41ad-9101-e6463c9ada36 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.767454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb43d2-c1bc-41ad-9101-e6463c9ada36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.768820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb43d2-c1bc-41ad-9101-e6463c9ada36 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:28.769312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb43d2-c1bc-41ad-9101-e6463c9ada36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:28.771544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb43d2-c1bc-41ad-9101-e6463c9ada36 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:28.780545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 172/346] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:28 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:28.782800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d17c-e44c-4b5a-b0f2-d387a967081c None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.783379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d17c-e44c-4b5a-b0f2-d387a967081c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.783823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9d17c-e44c-4b5a-b0f2-d387a967081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.788287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93761dd5-b8e0-4544-837d-c449671b1506 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:28.811844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-64c9d17c-e44c-4b5a-b0f2-d387a967081c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:28.813613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-64c9d17c-e44c-4b5a-b0f2-d387a967081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:28.836657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd775e-3e95-45dc-ad32-7362c50dbbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:28.837279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd775e-3e95-45dc-ad32-7362c50dbbf5 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.837552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd775e-3e95-45dc-ad32-7362c50dbbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.837821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd775e-3e95-45dc-ad32-7362c50dbbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.838504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd775e-3e95-45dc-ad32-7362c50dbbf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:28.838805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd775e-3e95-45dc-ad32-7362c50dbbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:28.840305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd775e-3e95-45dc-ad32-7362c50dbbf5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:28.841888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:28.850908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 175/347] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:28.852062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e06a-92d1-4a65-b4dd-487f2962c6c0 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.852411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e06a-92d1-4a65-b4dd-487f2962c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.852680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e06a-92d1-4a65-b4dd-487f2962c6c0 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.854128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 176/348] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:28.859272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61e06a-92d1-4a65-b4dd-487f2962c6c0 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:28.896859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 173/349] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:28.898303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba74b9a-fe07-4adc-8947-0ad586628701 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:28.898587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba74b9a-fe07-4adc-8947-0ad586628701 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:28.898857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba74b9a-fe07-4adc-8947-0ad586628701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:28.918366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-dba74b9a-fe07-4adc-8947-0ad586628701 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:28.919631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-dba74b9a-fe07-4adc-8947-0ad586628701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:28.930885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:28.969468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61e06a-92d1-4a65-b4dd-487f2962c6c0 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:29.004880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:29.005476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 174/350] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:29.012371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.012371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.012371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.084062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:29.090782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:29.092806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:29.193399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 177/351] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:29.201622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba74b9a-fe07-4adc-8947-0ad586628701 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:29.336025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba74b9a-fe07-4adc-8947-0ad586628701 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:29.360004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:29.372667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:29.373302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.373682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.373956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.375080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:29.376570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:29.378420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 175/352] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:29 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:29.386332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:29.422784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:29.532270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f14169-abe5-4dab-a624-56dfabef092e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:29.547440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:29.553565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8e6b7-801a-48a8-8ce7-a442042104f2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:29.593885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:29.594441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.594716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.594982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.609657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f9985-7b05-4180-a42c-7989675ed2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:29.610305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f9985-7b05-4180-a42c-7989675ed2f5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.610586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f9985-7b05-4180-a42c-7989675ed2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.610857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f9985-7b05-4180-a42c-7989675ed2f5 None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.612537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f9985-7b05-4180-a42c-7989675ed2f5 None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:29.614349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f9985-7b05-4180-a42c-7989675ed2f5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:29.615773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:29.616203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 176/353] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:29 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:29.617379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890575-085b-42a4-ae3f-8c54632d005f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.617656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890575-085b-42a4-ae3f-8c54632d005f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.617926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3890575-085b-42a4-ae3f-8c54632d005f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.619097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 177/354] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:29.619785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3890575-085b-42a4-ae3f-8c54632d005f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:29.624649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:29.625569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:29.627402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 178/355] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:29.628978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:29.737782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:29.763476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7336393-4321-4f4b-b31f-cad30ea98d23 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:29.765548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3890575-085b-42a4-ae3f-8c54632d005f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:29.805152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:29.805789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.806064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.806333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.807858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3882274-37d9-45d8-8114-114f4ce2506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:29.808529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3882274-37d9-45d8-8114-114f4ce2506e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.808802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3882274-37d9-45d8-8114-114f4ce2506e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.809067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3882274-37d9-45d8-8114-114f4ce2506e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.810498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:29.810846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:29.812609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:29.819544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 179/356] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:29 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:29.820494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:29.820801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3882274-37d9-45d8-8114-114f4ce2506e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-b3882274-37d9-45d8-8114-114f4ce2506e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:29.826811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:29.829062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 178/357] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:29.830984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d1aef-fac1-44e7-b8fe-f7bf988b368e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:29.832819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d1aef-fac1-44e7-b8fe-f7bf988b368e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:29.832819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d1aef-fac1-44e7-b8fe-f7bf988b368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:29.849577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:29.854327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8f4d1aef-fac1-44e7-b8fe-f7bf988b368e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:29.855895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8f4d1aef-fac1-44e7-b8fe-f7bf988b368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:29.946634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 179/358] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:29.947964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d1aef-fac1-44e7-b8fe-f7bf988b368e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:29.994032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b63d8d-e296-4edd-8dee-d235304efec0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:30.041832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:30.046197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ae553-1541-4d15-8609-c0527ab54635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:30.046826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ae553-1541-4d15-8609-c0527ab54635 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.047284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ae553-1541-4d15-8609-c0527ab54635 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.047603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ae553-1541-4d15-8609-c0527ab54635 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.048307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ae553-1541-4d15-8609-c0527ab54635 None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:30.048614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ae553-1541-4d15-8609-c0527ab54635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:30.050798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ae553-1541-4d15-8609-c0527ab54635 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:30.061407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 180/359] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:30.062772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ebc72-8c57-4a83-90bb-eeb10fc51420 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.063049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ebc72-8c57-4a83-90bb-eeb10fc51420 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.063321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ebc72-8c57-4a83-90bb-eeb10fc51420 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.068978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 181/360] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:30.070468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ebc72-8c57-4a83-90bb-eeb10fc51420 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:30.076876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d1aef-fac1-44e7-b8fe-f7bf988b368e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:30.121665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9efa01-3a87-4ebe-ac05-86ad43832a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:30.122249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9efa01-3a87-4ebe-ac05-86ad43832a02 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.122520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9efa01-3a87-4ebe-ac05-86ad43832a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.122779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9efa01-3a87-4ebe-ac05-86ad43832a02 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.124186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9efa01-3a87-4ebe-ac05-86ad43832a02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:30.124505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9efa01-3a87-4ebe-ac05-86ad43832a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:30.125910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9efa01-3a87-4ebe-ac05-86ad43832a02 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:30.143511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 180/361] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:30.144575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71d6fe-e933-473e-91f9-550b45792462 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.144851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71d6fe-e933-473e-91f9-550b45792462 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.145116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71d6fe-e933-473e-91f9-550b45792462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.167626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5d71d6fe-e933-473e-91f9-550b45792462 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:30.168836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5d71d6fe-e933-473e-91f9-550b45792462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:30.234126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:30.268109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ebc72-8c57-4a83-90bb-eeb10fc51420 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:30.284117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 181/362] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:30.285075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71d6fe-e933-473e-91f9-550b45792462 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:30.333372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:30.335591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.335591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.335591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.411940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:30.413573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:30.445810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:30.501570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:30.523628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 182/363] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:37:30.524528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:30.526995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71d6fe-e933-473e-91f9-550b45792462 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:30.609759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:30.610375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.610645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.610910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.634039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c492-5e2b-40e2-889b-4b38ae5674c2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:30.672455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:30.673010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.673282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.673548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.684527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:30.689868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:30.690973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 10:37:30.691996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:30.694530 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 183/364] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:30.695437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:30.741339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:30.742932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:30.743884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:30.813829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:30.826262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257c954-3e41-49ce-aa21-5381ce83dfe1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:30.864237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 182/365] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:30.865321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:30.867661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:30.868669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:30.869090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:30.869504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:30.870854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:30.872468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:30.874457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 184/366] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:30 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:30.879538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:30.984459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21508a3a-4875-4563-9a6b-a43c90330a78 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:30.995732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c0a89-45c3-4c2d-b238-e1f764d8f366 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.014248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:31.038349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.039761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.040197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.040818 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.041998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:31.042611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.044501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.051781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.052459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.052733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.053001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.059368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 183/367] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.060684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.068463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:31.074151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:31.075090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 185/368] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:31.075967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.106068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:31.193556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e33b0-6cae-4798-9b7f-8bbfce392f05 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.198466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c00b6b-d063-4630-ac89-0179d12a78ff None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.233969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.234681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.234931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.235159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.236315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.237795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.238637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.239415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 184/369] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.243939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.244860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.245384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.245862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.247448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.249189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.252473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 186/370] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.253493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.298274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:31.356036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:31.367640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3c773-1829-456c-8d6c-bb7db7f0107a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.381393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b794fe-9e64-456e-9739-8b7a63561e5d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.411360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.412112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.412438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.412717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] PATH_INFO: `/v3/users/2eafc41f6c0043678e5d33e010eecc76` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.413855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] RBAC: Authorizing `identity:get_user(user_id=2eafc41f6c0043678e5d33e010eecc76)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.415310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.417221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 185/371] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/users/2eafc41f6c0043678e5d33e010eecc76 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.417808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.423939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.424890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.425377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.425837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.426687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:31.427166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.428764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.439756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 187/372] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.445418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.546994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb0c24-2494-49ab-81be-e67d54d931f3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.568536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:31.573116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cabcd-c60f-49f6-a816-f4686a82e191 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.582717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e944c-a851-4d1e-ade3-f20285fadf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.583353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e944c-a851-4d1e-ade3-f20285fadf58 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.583629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e944c-a851-4d1e-ade3-f20285fadf58 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.583900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e944c-a851-4d1e-ade3-f20285fadf58 None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.587143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e944c-a851-4d1e-ade3-f20285fadf58 None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.588633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e944c-a851-4d1e-ade3-f20285fadf58 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.590222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 186/373] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.596409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1abc3b-8a91-4aec-8be0-3bebd9f9a832 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.596695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1abc3b-8a91-4aec-8be0-3bebd9f9a832 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.597098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1abc3b-8a91-4aec-8be0-3bebd9f9a832 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.598312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 187/374] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:31.602686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1abc3b-8a91-4aec-8be0-3bebd9f9a832 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.628812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.629938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.630213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.630479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.631559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=1840a5c96dff47be87e068418ad64f3c) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:31.631863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.633361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.634154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:31.639913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 188/375] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.640586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.737883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1abc3b-8a91-4aec-8be0-3bebd9f9a832 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.777211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.777834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.778114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.778375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.779272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] RBAC: Adding query filter params (role.id=8183b92ce1a644b7b561153ffa9f474d, user.id=2eafc41f6c0043678e5d33e010eecc76, scope.system=all) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:31.779575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.781290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:31.790532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d2761-ffe9-438d-a403-6123fe45a644 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:31.800774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 188/376] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/role_assignments?user.id=2eafc41f6c0043678e5d33e010eecc76&scope.system=all&role.id=8183b92ce1a644b7b561153ffa9f474d => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:31.809875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:31.831021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a2329-45be-4114-ac30-eed8270ead18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:31.831651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2329-45be-4114-ac30-eed8270ead18 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:31.831981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2329-45be-4114-ac30-eed8270ead18 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:31.832252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2329-45be-4114-ac30-eed8270ead18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:31.869041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:31.914023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a2329-45be-4114-ac30-eed8270ead18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:31.915652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a2329-45be-4114-ac30-eed8270ead18 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:32.008352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:32.028831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 189/377] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:32.030453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684fa1-e6eb-41bd-9b30-d967c4813310 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.030958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684fa1-e6eb-41bd-9b30-d967c4813310 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.031251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684fa1-e6eb-41bd-9b30-d967c4813310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.031496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e275f9-214e-4140-8792-32f3fa227001 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:32.059880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8d684fa1-e6eb-41bd-9b30-d967c4813310 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:32.061271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8d684fa1-e6eb-41bd-9b30-d967c4813310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:32.082210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:32.083146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.083425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.083682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.084446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=1840a5c96dff47be87e068418ad64f3c) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:32.084747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:32.086679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:32.094920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 189/378] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:31 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 260 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:32.096030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:32.152216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:32.167172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 190/379] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:32.214346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb89c7-7962-4960-a321-239bf5bb59d5 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.214859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb89c7-7962-4960-a321-239bf5bb59d5 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.215303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb89c7-7962-4960-a321-239bf5bb59d5 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.217103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 191/380] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:32.223289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7990d-38e6-42a1-9f3b-2cf0e5a4e814 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.223922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7990d-38e6-42a1-9f3b-2cf0e5a4e814 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.225164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7990d-38e6-42a1-9f3b-2cf0e5a4e814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.257370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-07d7990d-38e6-42a1-9f3b-2cf0e5a4e814 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:32.258557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-07d7990d-38e6-42a1-9f3b-2cf0e5a4e814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:32.266120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e980ff5-8d5e-4c58-9509-3b4694afd2a5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:32.309017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:32.310105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.310662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.311266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] PATH_INFO: `/v3/projects/1b6e50f80fbe471b845ef361f068a613` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.313754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] RBAC: Authorizing `identity:get_project(project_id=1b6e50f80fbe471b845ef361f068a613)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:32.315459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:32.317764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 190/381] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:32 2022] GET /identity/v3/projects/1b6e50f80fbe471b845ef361f068a613 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:32.332024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:32.364024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 192/382] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:32.370201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:32.462075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b63afd-4103-4af7-b463-07acdb23599b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:32.499082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:32.499721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.499997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.500281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] PATH_INFO: `/v3/groups/f583b5b638d243339885e602e06f35ea` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.507350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] RBAC: Authorizing `identity:get_group(group_id=f583b5b638d243339885e602e06f35ea)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:32.508854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:32.510551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 191/383] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:32 2022] GET /identity/v3/groups/f583b5b638d243339885e602e06f35ea => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:32.517043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:32.517212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7990d-38e6-42a1-9f3b-2cf0e5a4e814 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:32.624688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:32.625835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7990d-38e6-42a1-9f3b-2cf0e5a4e814 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:32.651162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe3425-2080-444a-a085-6a7e85e7ff74 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:32.664165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:32.664848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.665125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.665392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.666124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:32.677610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:32.683022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:32.683896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:32.685133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 193/384] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:32.690406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:32.708569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:32.709298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.709579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.709847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.710692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] RBAC: Adding query filter params (group.id=f583b5b638d243339885e602e06f35ea, role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1b6e50f80fbe471b845ef361f068a613) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:32.710997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:32.712510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:32.722593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 192/385] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:32 2022] GET /identity/v3/role_assignments?group.id=f583b5b638d243339885e602e06f35ea&scope.project.id=1b6e50f80fbe471b845ef361f068a613&role.id=e65d667683404c438483a1b1252ff2bb => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:32.759588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:32.823740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ffac2-ab1d-4586-bbfe-1cf22a74a4b2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:32.868499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:32.869125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:32.869401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:32.869664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:32.870347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:32.870644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:32.872081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:32.883141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 194/386] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:32.889174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:32.942677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:32.973990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd34b53-ac54-4c8c-8058-4560090a3a3c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.016721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.017773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.018244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.018728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.036025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:33.042854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:33.043767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:33.044618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:33.045296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06eee3-4834-40e9-8f61-5e413e65f8b1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.052212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 193/387] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:33.054921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:33.065165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:33.114158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.115124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.115561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.115997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.119879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:33.121619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:33.124124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 195/388] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:32 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:33.130000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:33.167751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13010d3a-7698-4ca7-8337-fd07c7242e2d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.204141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.204141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.204141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.204141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.205120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:33.205516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:33.207570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:33.220716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 194/389] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:33.223286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:33.294857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:33.296183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5b2e6-5f89-476a-bbdc-f828b6e7fd74 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.315371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:33.338919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.339660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.339937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.340208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.343502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:33.343631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:33.345337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 196/390] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:33 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:33.349734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf0e82-21a3-407d-9066-ca0f2c8ce7e2 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.351266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:33.386788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.388448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.389105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.389595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.391043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:33.392761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:33.399194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 195/391] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:33 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:33.400799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:33.479124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b9a63-1335-4793-861c-44a8bf2dc08d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.521480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.524041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.526001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.526485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.529987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9d09bb1714d98af3bd512e75cfe30, group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:33.533404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:33.559195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 197/392] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Aug 12 10:37:33 2022] PUT /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:37:33.567390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af8815-79f8-42c5-8d63-94d58acbcbad None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.617041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18967a-51c3-429b-a331-c3aeeb601a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.618116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18967a-51c3-429b-a331-c3aeeb601a10 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.618591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18967a-51c3-429b-a331-c3aeeb601a10 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.619066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18967a-51c3-429b-a331-c3aeeb601a10 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.620699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18967a-51c3-429b-a331-c3aeeb601a10 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:33.622444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18967a-51c3-429b-a331-c3aeeb601a10 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:33.624450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 196/393] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:33 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:33.630848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:33.697450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:33.770213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc69d2-d586-4640-88a8-936c41aef31c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:33.813274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348272df-28a9-4735-9829-ac26d53cfcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:33.814302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348272df-28a9-4735-9829-ac26d53cfcdb None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:33.814801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348272df-28a9-4735-9829-ac26d53cfcdb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:33.815284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348272df-28a9-4735-9829-ac26d53cfcdb None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:33.817672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348272df-28a9-4735-9829-ac26d53cfcdb None admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9d09bb1714d98af3bd512e75cfe30, group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:33.819329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348272df-28a9-4735-9829-ac26d53cfcdb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:33.836217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 198/394] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Aug 12 10:37:33 2022] PUT /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:34.152413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b013b-9962-45f7-93ae-5f85822fdd19 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:34.152600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b013b-9962-45f7-93ae-5f85822fdd19 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:34.152877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b013b-9962-45f7-93ae-5f85822fdd19 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:34.154221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 197/395] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:34.160125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d937-bc4f-4ef5-98ea-4d999dadf1e3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:34.160450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d937-bc4f-4ef5-98ea-4d999dadf1e3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:34.160725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d937-bc4f-4ef5-98ea-4d999dadf1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:34.185905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a8b4d937-bc4f-4ef5-98ea-4d999dadf1e3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:34.187158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a8b4d937-bc4f-4ef5-98ea-4d999dadf1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:34.275283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 199/396] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:34.550319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b013b-9962-45f7-93ae-5f85822fdd19 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:34.554662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:34.656424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b013b-9962-45f7-93ae-5f85822fdd19 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:34.690785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:34.691416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:34.691690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:34.691962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:34.702506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:34.707279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:34.708165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:34.709023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 198/397] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:34.714525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4d937-bc4f-4ef5-98ea-4d999dadf1e3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:34.746792 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:34.830236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4d937-bc4f-4ef5-98ea-4d999dadf1e3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:34.872541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0625722-7fe6-4f0b-ac47-7c9ee4d56d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:34.873188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625722-7fe6-4f0b-ac47-7c9ee4d56d3c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:34.873463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625722-7fe6-4f0b-ac47-7c9ee4d56d3c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:34.873781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625722-7fe6-4f0b-ac47-7c9ee4d56d3c None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:34.874452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0625722-7fe6-4f0b-ac47-7c9ee4d56d3c None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:34.874747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0625722-7fe6-4f0b-ac47-7c9ee4d56d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:34.876196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0625722-7fe6-4f0b-ac47-7c9ee4d56d3c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:34.888097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 200/398] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:34.893957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:34.980004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:35.019230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a9ca4-0690-4779-aedc-4f46f7115ff7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:35.044052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd252b-9ec4-4e70-a635-65bc8085b6c7 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.044052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd252b-9ec4-4e70-a635-65bc8085b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.044052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd252b-9ec4-4e70-a635-65bc8085b6c7 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.044052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 201/399] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:35.047894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb08b6-947d-4daf-9d3e-9b4cb9cf0707 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.048096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb08b6-947d-4daf-9d3e-9b4cb9cf0707 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.048384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb08b6-947d-4daf-9d3e-9b4cb9cf0707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.086707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-05fb08b6-947d-4daf-9d3e-9b4cb9cf0707 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:35.093052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-05fb08b6-947d-4daf-9d3e-9b4cb9cf0707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:35.101677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:35.102327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.102600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.102866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.103987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:35.105463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:35.112215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 199/400] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:34 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:35.119850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:35.231574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 202/401] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:35.251438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34032ba-9a91-48c4-94e9-0c6c0fe13808 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:35.283665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:35.288782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0b21e-d64c-4043-90f3-71c2fed27677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:35.289412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b21e-d64c-4043-90f3-71c2fed27677 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.289742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b21e-d64c-4043-90f3-71c2fed27677 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.290015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b21e-d64c-4043-90f3-71c2fed27677 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.291135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0b21e-d64c-4043-90f3-71c2fed27677 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:35.292806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0b21e-d64c-4043-90f3-71c2fed27677 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:35.295377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 200/402] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:35 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:35.300954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb08b6-947d-4daf-9d3e-9b4cb9cf0707 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:35.358341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:35.461803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb08b6-947d-4daf-9d3e-9b4cb9cf0707 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:35.510958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:35.511889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.512900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.513313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.514326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a74473-a9d4-41a9-915d-e75ad7a94f33 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.514722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a74473-a9d4-41a9-915d-e75ad7a94f33 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.515095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a74473-a9d4-41a9-915d-e75ad7a94f33 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.515889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1840a5c96dff47be87e068418ad64f3c, group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:35.516942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 201/403] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:35.517684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:35.522265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132c75-d31a-43dd-9554-fbcea9428198 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.522623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132c75-d31a-43dd-9554-fbcea9428198 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.522990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132c75-d31a-43dd-9554-fbcea9428198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.541067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 203/404] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Aug 12 10:37:35 2022] PUT /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:35.544539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-6b132c75-d31a-43dd-9554-fbcea9428198 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:35.546041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6b132c75-d31a-43dd-9554-fbcea9428198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:35.599362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:35.623768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 202/405] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:35.719857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:35.740477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b70890-5757-416e-885e-8271b1e7f4f5 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:35.790259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:35.790949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.791228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.791494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:35.803412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:35.808191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:35.809129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:35.809980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 204/406] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:35.815468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b132c75-d31a-43dd-9554-fbcea9428198 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:35.933030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:35.934218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b132c75-d31a-43dd-9554-fbcea9428198 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:35.985483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:35.998827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:35.999435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:35.999702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:35.999959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.000664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:36.000958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:36.002326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:36.017224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 203/407] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:36.023680 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.086948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c5d2d-b258-431e-b76a-3d813f5b7212 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.124715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea945fd-a3e2-450d-8b6e-8f7e66ddfc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.125350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea945fd-a3e2-450d-8b6e-8f7e66ddfc2e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.125628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea945fd-a3e2-450d-8b6e-8f7e66ddfc2e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.125899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea945fd-a3e2-450d-8b6e-8f7e66ddfc2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.136822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea945fd-a3e2-450d-8b6e-8f7e66ddfc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-bea945fd-a3e2-450d-8b6e-8f7e66ddfc2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:36.142735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:36.143617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:36.144824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 205/408] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:36.145648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85742fb3-85d2-4884-9b68-6a200633bbcb None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.145648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85742fb3-85d2-4884-9b68-6a200633bbcb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.145970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85742fb3-85d2-4884-9b68-6a200633bbcb None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.151209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 206/409] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:36.151209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85742fb3-85d2-4884-9b68-6a200633bbcb None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.154874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:36.160369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c6d96-9542-48be-8e96-ad521bba0cd5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.196049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c56aa-dce3-4df2-b2eb-398ab81302ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.196645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c56aa-dce3-4df2-b2eb-398ab81302ec None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.196920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c56aa-dce3-4df2-b2eb-398ab81302ec None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.197189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c56aa-dce3-4df2-b2eb-398ab81302ec None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.198309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c56aa-dce3-4df2-b2eb-398ab81302ec None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:36.199751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c56aa-dce3-4df2-b2eb-398ab81302ec None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:36.201707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 204/410] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:36 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:36.202837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14909e8-eee3-4d22-a48d-6c4105cf53dc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.203115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14909e8-eee3-4d22-a48d-6c4105cf53dc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.203384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14909e8-eee3-4d22-a48d-6c4105cf53dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.221583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a14909e8-eee3-4d22-a48d-6c4105cf53dc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:36.222763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a14909e8-eee3-4d22-a48d-6c4105cf53dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:36.258206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85742fb3-85d2-4884-9b68-6a200633bbcb None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.280238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:36.291539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 205/411] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:36.292416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14909e8-eee3-4d22-a48d-6c4105cf53dc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.305554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.306489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.306956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.307416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.308296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:36.308840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:36.310572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:36.325220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 207/412] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:36.332318 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.460252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14909e8-eee3-4d22-a48d-6c4105cf53dc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.472620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:36.494459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.495154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.495427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.495693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.496850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:36.498354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:36.500101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 206/413] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:36 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:36.505846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.543741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d2fb3-fcc4-4ec9-a983-b5e3532f32d7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.581317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a172509-4449-4bf5-9d9f-a9e352b0c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.581945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a172509-4449-4bf5-9d9f-a9e352b0c75d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.582221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a172509-4449-4bf5-9d9f-a9e352b0c75d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.582491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a172509-4449-4bf5-9d9f-a9e352b0c75d None admin] PATH_INFO: `/v3/groups/f583b5b638d243339885e602e06f35ea` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.583588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a172509-4449-4bf5-9d9f-a9e352b0c75d None admin] RBAC: Authorizing `identity:get_group(group_id=f583b5b638d243339885e602e06f35ea)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:36.585084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a172509-4449-4bf5-9d9f-a9e352b0c75d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:36.585459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:36.588316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 208/414] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:36 2022] GET /identity/v3/groups/f583b5b638d243339885e602e06f35ea => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:36.588316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3f597-2afe-4011-a773-cb676139d270 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.588316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3f597-2afe-4011-a773-cb676139d270 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.588567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3f597-2afe-4011-a773-cb676139d270 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.593827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 209/415] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:36.593827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3f597-2afe-4011-a773-cb676139d270 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.634592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7047dd9-ac70-4415-bddb-9e6884cbf8ad None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.669760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.670474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.670743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.671003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.673169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1840a5c96dff47be87e068418ad64f3c, group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:36.674602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:36.687912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 207/416] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Aug 12 10:37:36 2022] PUT /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:37:36.688736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.708442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3f597-2afe-4011-a773-cb676139d270 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.748578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2f200-b78c-4928-b1a9-955ea19f420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.750224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2f200-b78c-4928-b1a9-955ea19f420e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.750569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2f200-b78c-4928-b1a9-955ea19f420e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.750840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2f200-b78c-4928-b1a9-955ea19f420e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.753658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:36.767181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2f200-b78c-4928-b1a9-955ea19f420e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-d1c2f200-b78c-4928-b1a9-955ea19f420e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:36.772618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:36.773602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:36.774392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 210/417] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:36.775536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef0db-5006-4908-af60-b08c85c091dd None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.775630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef0db-5006-4908-af60-b08c85c091dd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.775852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef0db-5006-4908-af60-b08c85c091dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.796396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-18aef0db-5006-4908-af60-b08c85c091dd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:36.797601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-18aef0db-5006-4908-af60-b08c85c091dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:36.853135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693d5a-da06-4a76-9bb3-052371cb67d8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:36.878158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:36.883438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 211/418] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:36.884532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef0db-5006-4908-af60-b08c85c091dd None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:36.909661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:36.910295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:36.910565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:36.912301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] PATH_INFO: `/v3/projects/1b6e50f80fbe471b845ef361f068a613` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:36.914663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] RBAC: Authorizing `identity:get_project(project_id=1b6e50f80fbe471b845ef361f068a613)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:36.917178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:36.921819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 208/419] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:36 2022] GET /identity/v3/projects/1b6e50f80fbe471b845ef361f068a613 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:36.941573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:37.081738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef0db-5006-4908-af60-b08c85c091dd None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:37.122980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855a639-4d1b-410f-8872-ff12d8f2326e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:37.126802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:37.146348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:37.146952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.147223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.147483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.148141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:37.148463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:37.149837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:37.158677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 212/420] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:37.164126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:37.183816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:37.184890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.185357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.185939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] PATH_INFO: `/v3/projects/1b6e50f80fbe471b845ef361f068a613/groups/f583b5b638d243339885e602e06f35ea/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.188087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b6e50f80fbe471b845ef361f068a613, group_id=f583b5b638d243339885e602e06f35ea, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:37.189750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:37.207264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 209/421] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Aug 12 10:37:36 2022] PUT /identity/v3/projects/1b6e50f80fbe471b845ef361f068a613/groups/f583b5b638d243339885e602e06f35ea/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:37.208543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:37.272912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:37.334410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ccf1d-50cc-47de-9dc6-490f82084534 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:37.375168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:37.376205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d86d77-b3c7-46c2-a323-6302170974f0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:37.376343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.376797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.377262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.390006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:37.396302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:37.397168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:37.397978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:37.400212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 210/422] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:37.409334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:37.425761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:37.426372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.426644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.426913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.516182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:37.531026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:37.533862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:37.560346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a476a-f9ed-41dd-b810-2f7299c1932a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:37.572581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:37.626145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:37.626661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.626933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.627196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.627858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:37.628152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:37.629540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:37.652062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 211/423] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:37.657305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:37.700006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 213/424] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:37:37.707461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:37.831447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75803c-17bf-44ed-9fec-f7a7069569ba None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:37.833041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:37.850262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d53cc-2bf6-4fec-80d1-462a8f27744a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:37.873061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74ed41-d344-48f1-9fd9-46f99cfd4bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:37.873572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74ed41-d344-48f1-9fd9-46f99cfd4bc7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.874046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74ed41-d344-48f1-9fd9-46f99cfd4bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.874121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74ed41-d344-48f1-9fd9-46f99cfd4bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.905776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe91a6-2bdd-4cc7-bec1-9c7cd0da0bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:37.907362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe91a6-2bdd-4cc7-bec1-9c7cd0da0bb9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.908083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe91a6-2bdd-4cc7-bec1-9c7cd0da0bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.908570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe91a6-2bdd-4cc7-bec1-9c7cd0da0bb9 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.910150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe91a6-2bdd-4cc7-bec1-9c7cd0da0bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:37.911644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:37.914008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe91a6-2bdd-4cc7-bec1-9c7cd0da0bb9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:37.916361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 214/425] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:37 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:37.918808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5960b-e675-4e58-81f1-aed762d84fde None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:37.919292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5960b-e675-4e58-81f1-aed762d84fde None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:37.919786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5960b-e675-4e58-81f1-aed762d84fde None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:37.926134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 215/426] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:37.926134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5960b-e675-4e58-81f1-aed762d84fde None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:37.956053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74ed41-d344-48f1-9fd9-46f99cfd4bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:37.957281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74ed41-d344-48f1-9fd9-46f99cfd4bc7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:38.061514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 212/427] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:37:38.062831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151d742-9ebf-4143-9900-c263e7fb1e10 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.063112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151d742-9ebf-4143-9900-c263e7fb1e10 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.063377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151d742-9ebf-4143-9900-c263e7fb1e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.073730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5960b-e675-4e58-81f1-aed762d84fde None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:38.083163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e151d742-9ebf-4143-9900-c263e7fb1e10 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:38.084103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e151d742-9ebf-4143-9900-c263e7fb1e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:38.114343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:38.115101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.115370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.115634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.116853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:38.118447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:38.119978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 216/428] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:37 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:38.125290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:38.151897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:38.169358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:38.183511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 213/429] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:38.183511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151d742-9ebf-4143-9900-c263e7fb1e10 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:38.270218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06a4ff-1b02-400a-b6b1-3fdcbc99db62 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:38.334039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:38.334672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.334941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.335205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] PATH_INFO: `/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.336786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] RBAC: Authorizing `identity:get_project(project_id=5ba9d09bb1714d98af3bd512e75cfe30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:38.338231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:38.340741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 217/430] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:38 2022] GET /identity/v3/projects/5ba9d09bb1714d98af3bd512e75cfe30 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:38.347289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:38.360010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151d742-9ebf-4143-9900-c263e7fb1e10 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:38.454079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:38.463482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13e4e-0945-41c2-a0b9-14dc7b681592 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:38.503058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:38.504187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.504644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.505048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.506365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:38.506838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:38.507561 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.507960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.508431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.508651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:38.509340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:38.509739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:38.510945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 218/431] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:38 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:38.511330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:38.512122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:38.519600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 214/432] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:38 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 611 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:38.529227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:38.637783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d7aac-2546-4026-97ce-4d01c51437fe None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:38.664646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:38.682369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:38.683507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.683965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.684426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.697828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:38.704247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:38.705203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 10:37:38.706033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:38.709405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 219/433] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:38.719779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:38.752116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de810739-3e81-4e5d-9885-9c131fa07983 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:38.760202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:38.804486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd263bad-ca98-403c-b5b0-8adf580f6f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:38.805747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263bad-ca98-403c-b5b0-8adf580f6f44 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.806208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263bad-ca98-403c-b5b0-8adf580f6f44 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.806657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263bad-ca98-403c-b5b0-8adf580f6f44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.808253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd263bad-ca98-403c-b5b0-8adf580f6f44 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:38.808938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd263bad-ca98-403c-b5b0-8adf580f6f44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:38.811339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd263bad-ca98-403c-b5b0-8adf580f6f44 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:38.819948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 215/434] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:38 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=5ba9d09bb1714d98af3bd512e75cfe30&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 611 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:38.838557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d4a8d-f8e7-4b76-b191-9fde0f152329 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.838892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d4a8d-f8e7-4b76-b191-9fde0f152329 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.839104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d4a8d-f8e7-4b76-b191-9fde0f152329 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.840856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 216/435] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:38.847096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56c53-e079-484b-89f1-34b2c76f79c1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.847372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56c53-e079-484b-89f1-34b2c76f79c1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.847641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56c53-e079-484b-89f1-34b2c76f79c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.865241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714aac5-b6cb-4ece-b9aa-a7e651ec667a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:38.867969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-68c56c53-e079-484b-89f1-34b2c76f79c1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:38.868404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-68c56c53-e079-484b-89f1-34b2c76f79c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:38.911431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:38.912469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:38.912967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:38.913428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:38.914306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] RBAC: Adding query filter params (name=member) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:38.914830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:38.917373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:38.930874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 220/436] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:37:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:38.938436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:38.948799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 217/437] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:39.054130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:39.076679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb93dc5-1dd1-4871-aeb6-d7c34d8d8365 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:39.113761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:39.114637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.115045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.115447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.184865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:39.186702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:39.226187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c56c53-e079-484b-89f1-34b2c76f79c1 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.275316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:39.293608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:39.302273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 221/438] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:37:39.308520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.350989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c56c53-e079-484b-89f1-34b2c76f79c1 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:39.389314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378c9a2-95da-48fb-a123-1345d19bb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:39.389951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c9a2-95da-48fb-a123-1345d19bb42a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.390227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c9a2-95da-48fb-a123-1345d19bb42a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.390495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c9a2-95da-48fb-a123-1345d19bb42a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.403254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378c9a2-95da-48fb-a123-1345d19bb42a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-4378c9a2-95da-48fb-a123-1345d19bb42a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:39.408621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:39.409522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:39.410377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 218/439] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:39.413630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca42be-f139-4713-8fc2-2e9c6bfced22 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.414381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca42be-f139-4713-8fc2-2e9c6bfced22 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.414588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca42be-f139-4713-8fc2-2e9c6bfced22 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.415987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 219/440] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:39.416571 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca42be-f139-4713-8fc2-2e9c6bfced22 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.437398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da409d-f634-4b93-8d32-d905662a7aa9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:39.474681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64242eb1-760c-4a94-a535-0266c577feb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:39.475315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242eb1-760c-4a94-a535-0266c577feb4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.475589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242eb1-760c-4a94-a535-0266c577feb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.475858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242eb1-760c-4a94-a535-0266c577feb4 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.477041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64242eb1-760c-4a94-a535-0266c577feb4 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:39.478513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64242eb1-760c-4a94-a535-0266c577feb4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:39.480503 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 222/441] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:39.481662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0082b-abc4-40c7-b5e8-4a09ce3efd18 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.481937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0082b-abc4-40c7-b5e8-4a09ce3efd18 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.483362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0082b-abc4-40c7-b5e8-4a09ce3efd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.501757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1cc0082b-abc4-40c7-b5e8-4a09ce3efd18 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:39.502961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1cc0082b-abc4-40c7-b5e8-4a09ce3efd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:39.535986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca42be-f139-4713-8fc2-2e9c6bfced22 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:39.546059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:39.552800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:39.570180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 223/442] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:39.570932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0082b-abc4-40c7-b5e8-4a09ce3efd18 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.573056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:39.573923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.574264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.574600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.575329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:39.575687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:39.577444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:39.586321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 220/443] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:39.591645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.675293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0082b-abc4-40c7-b5e8-4a09ce3efd18 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:39.707780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda51b0e-91d9-4943-a4f3-a478fe225479 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:39.710779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:39.711471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.711745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.712035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.713163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:39.714606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:39.716356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 224/444] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:39.722054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.743029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:39.743755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.744124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.744512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.773773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:39.790283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:39.820311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:39.821691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:39.833344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1023f-0396-4d93-96cc-c612d7166a84 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:39.866805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:39.867485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:39.867659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:39.867895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:39.869661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=b96d83bc5d78463dbf0415bc66a0b0db, scope.project.id=1840a5c96dff47be87e068418ad64f3c) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:39.869661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:39.870737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:39.876595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 225/445] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=b96d83bc5d78463dbf0415bc66a0b0db => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:39.877365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.929796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 221/446] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:37:39.936236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:39.988150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab49159-beea-45dd-892f-3674812b2ad5 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.000798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:40.020837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:40.034135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.035070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.035352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.035914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.045353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a428d-f427-4601-a2ea-239829cb50b8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.046940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:40.052148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:40.053107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:37:40.053894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:40.054502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 226/447] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:40.060122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:40.085244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.086165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.086974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.087254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] PATH_INFO: `/v3/projects/1840a5c96dff47be87e068418ad64f3c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.088480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] RBAC: Authorizing `identity:get_project(project_id=1840a5c96dff47be87e068418ad64f3c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:40.090180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:40.092339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 222/448] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:39 2022] GET /identity/v3/projects/1840a5c96dff47be87e068418ad64f3c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:40.098204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:40.189490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aafe09-e4c0-4990-8aa4-c50dcf3469c2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.211876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2142f-0829-45f8-963a-a9f922973b86 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.233880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.234491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.234768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.235037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.235707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:40.236002 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:40.237425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:40.245682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 227/449] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:40.249909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.250626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.250905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.251173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] PATH_INFO: `/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.251300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:40.252310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] RBAC: Authorizing `identity:get_group(group_id=9cd5cdc3ca6b46c4a9f0e09e775d6e78)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:40.253773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:40.256317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 223/450] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:40 2022] GET /identity/v3/groups/9cd5cdc3ca6b46c4a9f0e09e775d6e78 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:40.261268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:40.277415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:40.283388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:40.351526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867671d-17f3-47c6-a6df-84527ee1f95e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.360957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116eca1-9a44-4f7f-b91e-68e25aee81c9 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.383921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.384550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.384824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.385095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.393911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.394625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.394900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.395165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.395884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] RBAC: Adding query filter params (group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78, role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=1840a5c96dff47be87e068418ad64f3c) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:40.396190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:40.397665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:40.403507 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 224/451] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:40 2022] GET /identity/v3/role_assignments?group.id=9cd5cdc3ca6b46c4a9f0e09e775d6e78&scope.project.id=1840a5c96dff47be87e068418ad64f3c&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:40.454957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:40.456497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:40.489930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:40.564122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 228/452] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 10:37:40.570264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:40.653211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:40.679379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf10e8c-0fbc-4883-961c-e013070e1e16 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.712788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.713397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.713727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.713964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] PATH_INFO: `/v3/projects/1b6e50f80fbe471b845ef361f068a613` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.715091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] RBAC: Authorizing `identity:get_project(project_id=1b6e50f80fbe471b845ef361f068a613)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:40.716561 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:40.718196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 225/453] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:37:40 2022] GET /identity/v3/projects/1b6e50f80fbe471b845ef361f068a613 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:40.723857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:40.826204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fd63f-0b4b-41a0-9667-46d108b2d641 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:40.860051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:40.866401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd58d3-4517-4628-a87e-a6f3a83abeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:40.867025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd58d3-4517-4628-a87e-a6f3a83abeb0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:40.867557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd58d3-4517-4628-a87e-a6f3a83abeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:40.867557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd58d3-4517-4628-a87e-a6f3a83abeb0 None admin] PATH_INFO: `/v3/groups/f583b5b638d243339885e602e06f35ea` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:40.868734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd58d3-4517-4628-a87e-a6f3a83abeb0 None admin] RBAC: Authorizing `identity:get_group(group_id=f583b5b638d243339885e602e06f35ea)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:40.870195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd58d3-4517-4628-a87e-a6f3a83abeb0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:40.871746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 229/454] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Aug 12 10:37:40 2022] GET /identity/v3/groups/f583b5b638d243339885e602e06f35ea => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:40.878439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:40.984040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10106c73-634e-43b7-b11b-32e4745914c0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:41.018228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410535c-7a1d-407d-a3bd-9b3f8464b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:41.018836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410535c-7a1d-407d-a3bd-9b3f8464b6fb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:41.019108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410535c-7a1d-407d-a3bd-9b3f8464b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:41.019375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410535c-7a1d-407d-a3bd-9b3f8464b6fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:41.020092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410535c-7a1d-407d-a3bd-9b3f8464b6fb None admin] RBAC: Adding query filter params (group.id=f583b5b638d243339885e602e06f35ea, role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1b6e50f80fbe471b845ef361f068a613) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:41.020425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410535c-7a1d-407d-a3bd-9b3f8464b6fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:41.021858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410535c-7a1d-407d-a3bd-9b3f8464b6fb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:41.027634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 226/455] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Aug 12 10:37:40 2022] GET /identity/v3/role_assignments?group.id=f583b5b638d243339885e602e06f35ea&scope.project.id=1b6e50f80fbe471b845ef361f068a613&role.id=e65d667683404c438483a1b1252ff2bb => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:43.284301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023347d1-8ebe-46f4-a85d-e8402a6d259c None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:43.284815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023347d1-8ebe-46f4-a85d-e8402a6d259c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:43.284888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023347d1-8ebe-46f4-a85d-e8402a6d259c None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:43.286444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 230/456] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:43.292718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7d3c0-2c95-47c1-9d4b-f45b958817b1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:43.293164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7d3c0-2c95-47c1-9d4b-f45b958817b1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:43.293255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7d3c0-2c95-47c1-9d4b-f45b958817b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:43.320230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d3a7d3c0-2c95-47c1-9d4b-f45b958817b1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:43.321449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d3a7d3c0-2c95-47c1-9d4b-f45b958817b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:43.324394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:43.418868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 227/457] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:43.538143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e44b-b8a8-4090-9745-e0c6e26b06e6 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:43.538314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e44b-b8a8-4090-9745-e0c6e26b06e6 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:43.538589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e44b-b8a8-4090-9745-e0c6e26b06e6 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:43.540213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 231/458] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:43.545814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44a47-efef-4397-8693-d1b8c5ba5ba2 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:43.546099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44a47-efef-4397-8693-d1b8c5ba5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:43.546375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e44a47-efef-4397-8693-d1b8c5ba5ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:43.570951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a2e44a47-efef-4397-8693-d1b8c5ba5ba2 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:43.572132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a2e44a47-efef-4397-8693-d1b8c5ba5ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:43.639472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc4772-f925-4739-8ae5-913f6b04242d None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:43.639472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc4772-f925-4739-8ae5-913f6b04242d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:43.639472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc4772-f925-4739-8ae5-913f6b04242d None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:43.640072 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 232/459] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:43.645533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b437c59-4b32-4d3e-be4f-cd01b460140e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:43.646146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b437c59-4b32-4d3e-be4f-cd01b460140e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:43.646604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b437c59-4b32-4d3e-be4f-cd01b460140e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:43.667683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2b437c59-4b32-4d3e-be4f-cd01b460140e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:43.669550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2b437c59-4b32-4d3e-be4f-cd01b460140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:43.684822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 228/460] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:43.742476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 233/461] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:43.763764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e44a47-efef-4397-8693-d1b8c5ba5ba2 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:43.860732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:43.904759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e44a47-efef-4397-8693-d1b8c5ba5ba2 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:43.946731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:43.947356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:43.947676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:43.947950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:43.962172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:43.963622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:43.981254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:43.982294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:43.983133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:43.983133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:43.983133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:43.983133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 229/462] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:43.988651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b437c59-4b32-4d3e-be4f-cd01b460140e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.024009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.138059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b437c59-4b32-4d3e-be4f-cd01b460140e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.148058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec383e1-5068-43e6-9cc0-e68f2fba7413 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.169151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:44.176551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.177613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:44.178122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.179241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.179782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.181030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:44.182427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.183839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.193188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.194119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.194612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.195066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.200337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 234/463] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:44.201560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.207151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.208912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:44.224951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:44.225897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:44.227058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:44.227058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:44.227058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:44.227058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 230/464] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:44.227972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.312076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fda08-371a-42ee-8335-9d55466a182e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.349476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeac23-7de7-44a3-b4a1-d311a18e3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.350123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeac23-7de7-44a3-b4a1-d311a18e3520 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.350412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeac23-7de7-44a3-b4a1-d311a18e3520 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.350625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeac23-7de7-44a3-b4a1-d311a18e3520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.352285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c467147-4e5c-480d-ac48-da1639f4bbdb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.362647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeac23-7de7-44a3-b4a1-d311a18e3520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.364278 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeac23-7de7-44a3-b4a1-d311a18e3520 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-29aeac23-7de7-44a3-b4a1-d311a18e3520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:44.380349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:44.381250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:44.382109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:44.382109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:44.382109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:44.382109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 235/465] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:44.382824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e291d3-3ee6-4147-b3f9-6dccb761dce2 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.382921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e291d3-3ee6-4147-b3f9-6dccb761dce2 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.383200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e291d3-3ee6-4147-b3f9-6dccb761dce2 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.384712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 236/466] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:44.385538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e291d3-3ee6-4147-b3f9-6dccb761dce2 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.394225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.394771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.395044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.395310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.395955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.397479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.420849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 231/467] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:44 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:44.429296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.440308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:44.466216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:44.510197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e291d3-3ee6-4147-b3f9-6dccb761dce2 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.549463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d181f3d-5a42-43d6-90f8-d333efa60f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.550407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d181f3d-5a42-43d6-90f8-d333efa60f58 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.550823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d181f3d-5a42-43d6-90f8-d333efa60f58 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.551275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d181f3d-5a42-43d6-90f8-d333efa60f58 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.551490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea3895-701b-42bc-892a-ebd760b1a4ce None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.552076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d181f3d-5a42-43d6-90f8-d333efa60f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:44.552599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d181f3d-5a42-43d6-90f8-d333efa60f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.554346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d181f3d-5a42-43d6-90f8-d333efa60f58 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.568614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 237/468] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:44.570231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c249-40ee-41cb-96eb-19e109fe12a9 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.570654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c249-40ee-41cb-96eb-19e109fe12a9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.571061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c249-40ee-41cb-96eb-19e109fe12a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.592826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7720c249-40ee-41cb-96eb-19e109fe12a9 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:44.594674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7720c249-40ee-41cb-96eb-19e109fe12a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:44.596436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.597048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.597326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.597707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.598357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:44.598650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.600043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.612818 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 232/469] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:44.614579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.684782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 238/470] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:44.689404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720c249-40ee-41cb-96eb-19e109fe12a9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:44.737022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:44.749321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4914db-3b8d-4823-bdcb-b833aaa3908b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.755042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:44.790568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.791179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.791453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.791726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.792411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.794122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.819820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 233/471] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:44 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:44.881826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720c249-40ee-41cb-96eb-19e109fe12a9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:44.926403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21f0bc-78f1-4f03-b897-a304774c7de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:44.927021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f0bc-78f1-4f03-b897-a304774c7de4 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:44.927390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f0bc-78f1-4f03-b897-a304774c7de4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:44.927659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f0bc-78f1-4f03-b897-a304774c7de4 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:44.928325 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21f0bc-78f1-4f03-b897-a304774c7de4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:44.929837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21f0bc-78f1-4f03-b897-a304774c7de4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:44.966021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 239/472] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:44 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:45.022956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:45.111476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440fa52-f637-4505-b237-3d1b93133faf None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:45.111803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440fa52-f637-4505-b237-3d1b93133faf None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:45.112024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440fa52-f637-4505-b237-3d1b93133faf None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:45.113665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 240/473] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:45.119532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023a5f5-cf1d-4989-bef2-714b8fca975d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:45.119827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023a5f5-cf1d-4989-bef2-714b8fca975d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:45.120076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023a5f5-cf1d-4989-bef2-714b8fca975d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:45.150354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6023a5f5-cf1d-4989-bef2-714b8fca975d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:45.151572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6023a5f5-cf1d-4989-bef2-714b8fca975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:45.174387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0da86-fa90-43bc-8c53-b688cdffbc0f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:45.225599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:45.226160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:45.226433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:45.226702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:45.240475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:45.241940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:45.242057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 241/474] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:45.252373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:45.264624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:45.265634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:45.266681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:45.266681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:45.266681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:45.268320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 234/475] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:45.271915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023a5f5-cf1d-4989-bef2-714b8fca975d None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:45.292109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:45.379035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023a5f5-cf1d-4989-bef2-714b8fca975d None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:45.415157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:45.415767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:45.416042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:45.416342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:45.416996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:45.417296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:45.418744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:45.434923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 242/476] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:45.436587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:45.561464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f400073-01e7-48e5-8d78-184243e6032c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:45.563994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:45.600410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:45.601024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:45.601295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:45.601563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:45.602224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:45.603683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:45.634078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 235/477] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:45 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:37:45.708096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:45.740609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7dae1-2388-4a8a-a525-311dd97c0b7f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:45.780583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:45.781196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:45.781471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:45.781742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:45.782474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:45.783873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:45.792541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.sql.core [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 12 10:37:45.792541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 12 10:37:45.792541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [parameters: {'id': '262f0f884bcd403baf13fe2634f8eac6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 12 10:37:45.792541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93138) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application conn.query(q) Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result.read() Aug 12 10:37:45.810810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return ref.to_dict() Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 10:37:45.811799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application next(self.gen) Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application yield resource Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application next(self.gen) Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.session.rollback() Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.force_reraise() Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise self.value Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application session.commit() Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 12 10:37:45.812779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._prepare_impl() Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.session.flush() Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._flush(objects) Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application compat.raise_( Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application flush_context.execute() Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rec.execute(self) Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 12 10:37:45.813657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 10:37:45.814600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application conn.query(q) Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result.read() Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 10:37:45.815403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [parameters: {'id': '262f0f884bcd403baf13fe2634f8eac6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:45.816167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 12 10:37:45.818933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:45.820795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 243/478] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Aug 12 10:37:45 2022] POST /identity/v3/roles => generated 144 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:45.826749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:45.890779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:45.940579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b53a5-ea4d-4eb0-bef3-7c0db4f14b1f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:45.975189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fe687-53fb-4b0f-9a5d-3eec3fff8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:45.976153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fe687-53fb-4b0f-9a5d-3eec3fff8498 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:45.976642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fe687-53fb-4b0f-9a5d-3eec3fff8498 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:45.977121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fe687-53fb-4b0f-9a5d-3eec3fff8498 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:45.988058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fe687-53fb-4b0f-9a5d-3eec3fff8498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-670fe687-53fb-4b0f-9a5d-3eec3fff8498 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:45.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:45.995452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 10:37:45.996236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:45.998926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 236/479] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:37:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:46.004836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:46.112731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d4281-ce6d-42df-9aea-6ddcbdd5fe48 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:46.150618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11196ce7-a08b-4c0a-91d3-13b316246668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:46.151561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196ce7-a08b-4c0a-91d3-13b316246668 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:46.152065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196ce7-a08b-4c0a-91d3-13b316246668 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:46.152573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196ce7-a08b-4c0a-91d3-13b316246668 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:46.153470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196ce7-a08b-4c0a-91d3-13b316246668 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:46.153996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196ce7-a08b-4c0a-91d3-13b316246668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:46.155770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196ce7-a08b-4c0a-91d3-13b316246668 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:46.165300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 244/480] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:46.627708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9907a-1718-40a5-b5f5-2f9e8201c32b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:46.628744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9907a-1718-40a5-b5f5-2f9e8201c32b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:46.629220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9907a-1718-40a5-b5f5-2f9e8201c32b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:46.630841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 237/481] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:46.639353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029de5c-be11-4b5b-879a-ac66bfeaabb4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:46.640388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029de5c-be11-4b5b-879a-ac66bfeaabb4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:46.640855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029de5c-be11-4b5b-879a-ac66bfeaabb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:46.650651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:46.664436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a029de5c-be11-4b5b-879a-ac66bfeaabb4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:46.666009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a029de5c-be11-4b5b-879a-ac66bfeaabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:46.761056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 245/482] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:47.097885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9907a-1718-40a5-b5f5-2f9e8201c32b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:47.232516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9907a-1718-40a5-b5f5-2f9e8201c32b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:47.239241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3ec4-9881-462a-b7d4-1f57bbbd05c1 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.239527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3ec4-9881-462a-b7d4-1f57bbbd05c1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.239799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3ec4-9881-462a-b7d4-1f57bbbd05c1 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.241166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 246/483] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:47.247240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:47.251302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d027868-70d6-4fcf-8372-de0053f090d3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.251579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d027868-70d6-4fcf-8372-de0053f090d3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.251847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d027868-70d6-4fcf-8372-de0053f090d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.276489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:47.277484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.277952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.278432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.288965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6d027868-70d6-4fcf-8372-de0053f090d3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:47.292160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6d027868-70d6-4fcf-8372-de0053f090d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:47.298491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:47.303389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:47.304282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:47.305093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:47.313133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 238/484] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:47.317491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:47.430308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 247/485] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:47.469215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee112b3-b08f-422a-9a03-dfc64f04376d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:47.517661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28c8a5-1561-421f-8bb6-c6206bdaa636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:47.518273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c8a5-1561-421f-8bb6-c6206bdaa636 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.518548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c8a5-1561-421f-8bb6-c6206bdaa636 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.518814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c8a5-1561-421f-8bb6-c6206bdaa636 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.519489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28c8a5-1561-421f-8bb6-c6206bdaa636 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:47.519784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28c8a5-1561-421f-8bb6-c6206bdaa636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:47.521184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28c8a5-1561-421f-8bb6-c6206bdaa636 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:47.537007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 239/486] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:47.538401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d027868-70d6-4fcf-8372-de0053f090d3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:47.630737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:47.669436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d027868-70d6-4fcf-8372-de0053f090d3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:47.707762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:47.708337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.708618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.708890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.719110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:47.720590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:47.725202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1c97f-e691-42fe-8e43-e34c2e76a99f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.726046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1c97f-e691-42fe-8e43-e34c2e76a99f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.726527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1c97f-e691-42fe-8e43-e34c2e76a99f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.729456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 240/487] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:47.733845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669788f-1ef9-4783-afcf-183b5c648918 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.734431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669788f-1ef9-4783-afcf-183b5c648918 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.735007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669788f-1ef9-4783-afcf-183b5c648918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:47.735517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:47.736689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:47.738533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:47.738533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:47.738533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:47.738533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 248/488] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:47.744378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:47.778043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d669788f-1ef9-4783-afcf-183b5c648918 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:47.779599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d669788f-1ef9-4783-afcf-183b5c648918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:47.809297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:47.850332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 241/489] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:47.851186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669788f-1ef9-4783-afcf-183b5c648918 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:47.867475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2da74-c3b7-4334-97ce-e28052a11794 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:47.885222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:47.905767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:47.906720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:47.907312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:47.907778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:47.908650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:47.909145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:47.910992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:47.924238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 249/490] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:47.935761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:47.960752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669788f-1ef9-4783-afcf-183b5c648918 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:47.996243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.005204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.005450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.005691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.024361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:48.035301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:48.036244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:48.037087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 242/491] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:48.045520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.080297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086aee33-f914-4979-b496-5707f85052b9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.099236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.129397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.130514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.130983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.131473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] PATH_INFO: `/v3/users/nova` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.140556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.142401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:48.156478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:48.157444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:37:48.158331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:48.160579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 250/492] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:37:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:48.172293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.199225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b07fd0-063d-44e1-9030-7ede5a5bfcd8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.212685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.239037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.239644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.239917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.240182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.240879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:48.241263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.242742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.256462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 243/493] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:48.257489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.280061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564a037-d49d-45b1-90b1-8390ef091def None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.316436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.317034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.317306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.317572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.318302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:48.318597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.320037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.353022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.362967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 251/494] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:48.362967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.373055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fefcbd-7858-4a65-9a2a-70f2cb93d5a4 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.411420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.412352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.413262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.413675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.428805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:48.435915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:48.437030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:48.437853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:48.440768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 244/495] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:48.441691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.464061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.475556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63efc0-adf8-4548-987f-2807a27f9c9a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.518078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.518947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.519363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.519786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.530464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.531949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:48.548458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:48.549420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:48.550263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:48.550263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:48.550263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:48.554699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 252/496] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:48.555576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.555729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc249d-40b5-4ab6-a540-5274af479064 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.592207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18e823-14e6-4e18-9e81-0a88519fbde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.592827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18e823-14e6-4e18-9e81-0a88519fbde5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.593095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18e823-14e6-4e18-9e81-0a88519fbde5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.593355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18e823-14e6-4e18-9e81-0a88519fbde5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.611531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.657445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42345cd3-e7a1-4532-886e-718ea22aee4f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.661132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18e823-14e6-4e18-9e81-0a88519fbde5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.662602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18e823-14e6-4e18-9e81-0a88519fbde5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.691621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3263-594f-4245-9344-ab028838e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.692246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3263-594f-4245-9344-ab028838e519 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.692544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3263-594f-4245-9344-ab028838e519 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.692814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3263-594f-4245-9344-ab028838e519 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.693463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f3263-594f-4245-9344-ab028838e519 None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:48.693761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f3263-594f-4245-9344-ab028838e519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.695117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f3263-594f-4245-9344-ab028838e519 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.703113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 253/497] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:48.703898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3263-594f-4245-9344-ab028838e519 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.715622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.760913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 245/498] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:48.762455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7397186-dec7-475c-9e2c-453bc27c90f9 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.762720 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7397186-dec7-475c-9e2c-453bc27c90f9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.763031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7397186-dec7-475c-9e2c-453bc27c90f9 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.768326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 246/499] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:48.768326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723725-f099-4420-8a8d-99834c072087 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.768326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723725-f099-4420-8a8d-99834c072087 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.768326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723725-f099-4420-8a8d-99834c072087 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.768326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 247/500] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:48.768326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723725-f099-4420-8a8d-99834c072087 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.808682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3263-594f-4245-9344-ab028838e519 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.815156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.840719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.841370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.841640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.841934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.842565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:48.842857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.844229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.855396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 254/501] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:48.856225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:48.866278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723725-f099-4420-8a8d-99834c072087 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.901625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33a62e-23cf-4f2a-a580-ac7f02673a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.902235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a62e-23cf-4f2a-a580-ac7f02673a23 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.902514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a62e-23cf-4f2a-a580-ac7f02673a23 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.902771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a62e-23cf-4f2a-a580-ac7f02673a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.913400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33a62e-23cf-4f2a-a580-ac7f02673a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:48.914830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33a62e-23cf-4f2a-a580-ac7f02673a23 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:48.923270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-da33a62e-23cf-4f2a-a580-ac7f02673a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:48.928876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:48.929893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:48.930739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:48.930739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:48.930739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:48.930739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 248/502] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:48.931208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceabbbf-4e46-4dc5-b35b-b85cbb46d4a3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.931328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceabbbf-4e46-4dc5-b35b-b85cbb46d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.931602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceabbbf-4e46-4dc5-b35b-b85cbb46d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:48.948846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0ceabbbf-4e46-4dc5-b35b-b85cbb46d4a3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:48.950029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0ceabbbf-4e46-4dc5-b35b-b85cbb46d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:48.956782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca7cc0-1af0-4fd4-a52f-23145b6c915f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:48.989211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:48.989744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:48.990012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:48.990279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.000271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.001737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.016302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 249/503] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:49.016967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:49.017824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:49.023425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:49.023425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:49.023425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:49.023590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6b909-40bc-44aa-9fd0-5d3635cbb246 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.023590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6b909-40bc-44aa-9fd0-5d3635cbb246 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.023590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6b909-40bc-44aa-9fd0-5d3635cbb246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.023771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 255/504] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:49.023771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.044524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-13b6b909-40bc-44aa-9fd0-5d3635cbb246 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:49.045694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-13b6b909-40bc-44aa-9fd0-5d3635cbb246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:49.069858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.112793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 250/505] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:49.113942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6b909-40bc-44aa-9fd0-5d3635cbb246 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.141573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edcc05-3fb0-4fb6-82f0-d78511f54c72 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.167229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.185994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.186849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.187255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.187652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.197982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.200315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.214926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6b909-40bc-44aa-9fd0-5d3635cbb246 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:49.215114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:49.216315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 12 10:37:49.217252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:49.220398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 256/506] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:49.221375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.256440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.257366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.257826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.258296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.259118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:49.259616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.261162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.273123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 251/507] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:49.278614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.306777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.345205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31b2e7-cdaa-49cd-a227-c69ecccfbdf2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.380250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.380887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.381166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.381403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.382036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:49.382336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.383709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.389063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3f05d-5467-4f8d-9682-5f77eea0559b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.396791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 257/508] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:49.397482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.427702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.428330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.428610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.428872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.429589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:49.429884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.431312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.438954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.468536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 252/509] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:49.469354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.494177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f87f3c-eff1-4871-8854-bab84ebbf148 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.526817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.527363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.527630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.527891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.534866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.536364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.546741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:49.549077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:49.550009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:49.550896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 258/510] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:49.551451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.571729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5d35-115a-4bc4-8142-ab928b71a465 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.606847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.607615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.607896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.608164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.618672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.620137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:49.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:49.635596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:49.636458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:49.636458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:49.636458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:49.636458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 253/511] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:49.636683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.649695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.652892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1d369-5f77-4e06-9ffa-2331e1fe1726 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.688223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.688876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.689140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.689399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.696125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.697604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:49.709608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:49.710449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:49.711295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 259/512] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:49.711780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.741492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f59781-71a8-4fa1-8072-003928630ce7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.769267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.775480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.776126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.776429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.776698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.786990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:49.792005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:49.793066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 254/513] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:49.793869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.816084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b671d57-70ed-413b-9a56-cfb3d3e203c1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.849461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.850063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.850324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.877977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:49.890853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920446b-0fcc-4444-8782-ee56fd109333 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:49.917782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.919302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.923284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:49.923881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:49.924154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:49.924461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:49.925177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:49.925476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:49.926900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:49.944842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 255/514] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:49.945688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:49.995253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.026414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 260/515] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:50.027207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.051083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6fa97-a19a-4a3b-8944-74092f18d101 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.085456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.085679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.086275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.086540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.086859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.087658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:50.088093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.089497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.100344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 256/516] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:50.101126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.127377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36644fa5-d1d2-4ec2-980c-fcaec3237979 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.162797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.163393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.163662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.163926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.164650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:50.164947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.166385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.188319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 261/517] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:50.188319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.201183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d020e-bc1b-4bfc-a1dc-20291d86ab07 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.210820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.236172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.236801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.237085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.237352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.238035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:50.238328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.239699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.247494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 257/518] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:50.248373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.286540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c217-1e19-4859-8083-b9e6d469d308 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.305734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.319755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.320383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.320654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.320919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.348994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2532eef-b1da-4693-8aa3-cea810bc3101 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.384189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.384839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.385111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.385374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.394279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.395757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.396543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.397986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:50.412695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:50.413611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:50.414662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:50.414662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:50.414662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:50.414662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 258/519] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:50.414662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.439594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.510209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 262/520] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:50.511055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.520520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccc9a3-9755-4622-bd8e-7074d4332992 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.535169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.562479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.563006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.563287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.563551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.564199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.565720 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.588301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 259/521] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:37:50 2022] POST /identity/v3/users => generated 247 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:37:50.588301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.629141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4ddbd-ee41-42ea-8d84-794820c54568 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.668611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.669781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.670275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.670751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.671607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a25194d34d67429b92944e0484de03cd) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:50.672008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.673843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.692287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 263/522] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/role_assignments?user.id=a25194d34d67429b92944e0484de03cd&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:50.692287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.761634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.800123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7deebc-cf5c-4787-b00d-b04d4d4c4be4 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.802603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:50.822529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9abb3-7e5e-432e-a78d-029a3c2eb384 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:50.839627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.840220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.840519 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.840780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.854430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.855919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.868060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:50.868718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:50.868988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:50.869249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:50.869891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:50.870190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:50.871610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:50.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:50.875334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:50.876182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:50.876182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:50.876182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:50.876508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 260/523] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:50.877691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.890730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 264/524] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:50.891899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:50.986027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f89bec-a835-4f3a-85c3-8975a7989996 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.010190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb27f5-c81f-470b-b6c5-2ca13380a0b0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.017393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.024900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.025619 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.025896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.026161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.037845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.039404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.046586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.047260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.047542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.047814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.048547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:51.048851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.050294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:51.055326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:51.057480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:51.059475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:51.059475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:51.059475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:51.059475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 261/525] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:51.062951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.063155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 265/526] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:51.067137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.067945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.169053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908aba9-6876-40d9-a444-67ee0df4f003 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.172695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ad273-778e-4fe9-8b94-2bc9b5eb6f2d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.203710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.204347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.204625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.204893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.205542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:51.205842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.207225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.208704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.209331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.209692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.209961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.217063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.218524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.220034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 266/527] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:51.220989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:51.235563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:51.236477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:51.237348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 262/528] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:51.237808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.291485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.319436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.329687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7811b-e7e3-48c7-8513-cd6542a63eac None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.353179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa31f3-b8ae-4924-a0ae-ae7dead2c9ac None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.364976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.365557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.365830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.366247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.373923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.375386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:51.387644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:51.388929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:51.389848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 267/529] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:51.390390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.393793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.394716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.395189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.395640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.403224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.405253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:51.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:51.418139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:51.418969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:51.421349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 263/530] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:51.426898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.490694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d890bdb-f96b-449f-b2d5-12bbb6c2e8ac None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.514472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.525081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.525746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.526018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.526284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.527028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:51.527334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.528881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.536224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194ebb-7444-47dd-8bcd-00129dcf8771 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.538040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.546196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 268/531] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:51.547124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.574674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.575671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.576328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.576784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.577786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:51.578319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.579953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.599559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 264/532] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:51.600788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.648619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b16c6-9e8b-4a51-b4df-bb7420a6915a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.684666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.685315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.685652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.685919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.686621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:51.686922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.688691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.704299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 269/533] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:51.704299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.717851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658b4f6-17c5-4614-adaa-e2917e683c1d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.752078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.758008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.758900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.759455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.759928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.760870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=8571f3d80b0749729012353fcbbd9e4b) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:51.761398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.763881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.770668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 265/534] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/role_assignments?user.id=8571f3d80b0749729012353fcbbd9e4b&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:51.771709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:51.801037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:51.862768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749e2d5-7405-463b-8bf9-1fef6e2baa5d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.891780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65363d71-cc90-4f4d-9589-45390231d2fe None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:51.929311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.930269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.930732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.931184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:51.932163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=76f4fb448f1443b7a77fd8e1954defe5) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:51.932749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:51.934484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:51.941455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 266/535] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/role_assignments?user.id=76f4fb448f1443b7a77fd8e1954defe5&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:51.967155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:51.967775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:51.968047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:51.968330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.095992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:52.097826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:52.147186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:52.204616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 270/536] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:52.213045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:52.334881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:52.350529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1ab73-b0b2-42b4-a656-8852a1fb5a19 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:52.385823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:52.386446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.386717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.387293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.398795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:52.400286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:52.414826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:52.415772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:52.416653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:52.416653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:52.416653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:52.416653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 267/537] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:52.425381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:52.540239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137f09-0cac-44a2-bd47-af23f530aa89 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:52.579790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563bcb5-dacb-4c35-b675-e618b866e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:52.580434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563bcb5-dacb-4c35-b675-e618b866e0cb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.580712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563bcb5-dacb-4c35-b675-e618b866e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.580981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563bcb5-dacb-4c35-b675-e618b866e0cb None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.581815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563bcb5-dacb-4c35-b675-e618b866e0cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:52.582116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563bcb5-dacb-4c35-b675-e618b866e0cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:52.583492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563bcb5-dacb-4c35-b675-e618b866e0cb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:52.600070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 271/538] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:52.605920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:52.745603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce5f8d-b181-4638-b088-d20b35f6a602 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:52.778497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93230e16-6d51-45cf-bc0f-66137c564cec None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.778805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93230e16-6d51-45cf-bc0f-66137c564cec None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.779044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93230e16-6d51-45cf-bc0f-66137c564cec None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.780376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:52.781000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.781273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.781539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.781969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 272/539] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:52.788543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:52.790104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:52.792713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:52.794279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873981-fab6-4249-be27-0e888e3ea09a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.794554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873981-fab6-4249-be27-0e888e3ea09a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.794823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873981-fab6-4249-be27-0e888e3ea09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:52.804002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:52.806277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:52.808290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 268/540] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:52.809376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:52.815221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:52.835515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ef873981-fab6-4249-be27-0e888e3ea09a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:52.836779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ef873981-fab6-4249-be27-0e888e3ea09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:52.927732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b07c2-bd07-4a3d-83e1-3e8065d6ca1f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:52.929777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 273/541] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:52.931990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca71c8a-e470-463c-b558-b82cdef6ac0d None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.932283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca71c8a-e470-463c-b558-b82cdef6ac0d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.932546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca71c8a-e470-463c-b558-b82cdef6ac0d None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.936350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 274/542] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:52.939214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3474-0f28-4f78-a838-43350e4ae98c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.939483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3474-0f28-4f78-a838-43350e4ae98c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.939745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f3474-0f28-4f78-a838-43350e4ae98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.958009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3a1f3474-0f28-4f78-a838-43350e4ae98c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:52.959185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3a1f3474-0f28-4f78-a838-43350e4ae98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:52.968892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:52.969900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:52.970360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:52.970847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:52.971781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:52.972305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:52.974109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:52.994046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 269/543] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:53.001423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:53.032301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 275/544] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:53.098702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:53.151407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a2889-eb4f-4557-845d-87c3afcdc220 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:53.203563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:53.204181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:53.204475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:53.204738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:53.205471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a8f3a56366d348bfb0a5c445905dc46b) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:53.205772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:53.207209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:53.213695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 270/545] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:53 2022] GET /identity/v3/role_assignments?user.id=a8f3a56366d348bfb0a5c445905dc46b&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:53.351871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f3474-0f28-4f78-a838-43350e4ae98c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:53.435191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:53.437403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:53.478050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f3474-0f28-4f78-a838-43350e4ae98c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:53.519802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:53.520454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:53.520772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:53.521567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:53.533000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:53.538677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:53.540710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:53.542502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 276/546] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:53.550212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:53.599846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6cd03-97f4-4ef4-ac4d-c8fd61078f89 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:53.633317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:53.655507 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:53.656249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:53.656546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:53.656811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:53.675048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:53.680554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:53.681419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:53.682200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:53.684442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 271/547] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:37:53.690938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:53.720286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77150d-067d-4858-8069-7c81428da6d2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:53.763168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:53.792968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:53.793618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:53.793861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:53.794182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:53.794965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:53.795265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:53.796698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:53.804491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b4c7a-c031-4f2e-bfb1-0d264f1ec232 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:53.813025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 277/548] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:53.817432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:53.840983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:53.841937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:53.842347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:53.842979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:53.843992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:53.844472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:53.846047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:53.855116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 272/549] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:53.863099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:53.963253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:53.965647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350d926-4280-4da8-a744-4f1a1c674233 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.017967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.018614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.018892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.019262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.038826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cca1d-eacb-453a-b70a-8381c1594602 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.039017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.048524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.076852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.077844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.078259 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.079081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:54.080317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:54.081557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:54.082622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:54.082622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:54.082622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:54.082622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 278/550] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:54.089557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.094791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.097344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:54.126558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:54.127739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:54.128808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:54.128808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:54.128808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:54.131440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 273/551] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:54.137570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.188669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:54.250426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522597c-bd5f-487c-9262-c0eac3f04024 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.284466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:54.287513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20076a55-5f32-4178-8b11-cbb86f2cf91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.288119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20076a55-5f32-4178-8b11-cbb86f2cf91a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.288460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20076a55-5f32-4178-8b11-cbb86f2cf91a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.288557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41115-0648-4dde-96dc-0585ce2a1e3a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.288723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20076a55-5f32-4178-8b11-cbb86f2cf91a None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.289379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20076a55-5f32-4178-8b11-cbb86f2cf91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:54.289679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20076a55-5f32-4178-8b11-cbb86f2cf91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.291057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20076a55-5f32-4178-8b11-cbb86f2cf91a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.303243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 279/552] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:54.304895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac18e0-d905-47ee-8c6d-3aaa7549e9f0 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.304895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac18e0-d905-47ee-8c6d-3aaa7549e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.305066 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac18e0-d905-47ee-8c6d-3aaa7549e9f0 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.310102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 280/553] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:54.311280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dac309-2cf4-4070-9c9b-6a9a5e33dd24 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.311558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dac309-2cf4-4070-9c9b-6a9a5e33dd24 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.311825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dac309-2cf4-4070-9c9b-6a9a5e33dd24 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.313176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 281/554] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:54.316378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dac309-2cf4-4070-9c9b-6a9a5e33dd24 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.326153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94fbb-9712-4dc7-9658-ad1105bb3851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.326700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94fbb-9712-4dc7-9658-ad1105bb3851 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.326975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94fbb-9712-4dc7-9658-ad1105bb3851 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.327246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94fbb-9712-4dc7-9658-ad1105bb3851 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.327889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94fbb-9712-4dc7-9658-ad1105bb3851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:54.328187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94fbb-9712-4dc7-9658-ad1105bb3851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.329622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94fbb-9712-4dc7-9658-ad1105bb3851 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.345887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 274/555] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:54.345887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5594-114c-4410-be6f-1c0fb523e66c None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.345887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5594-114c-4410-be6f-1c0fb523e66c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.345887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5594-114c-4410-be6f-1c0fb523e66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.361333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-964c5594-114c-4410-be6f-1c0fb523e66c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:54.362522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-964c5594-114c-4410-be6f-1c0fb523e66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:54.438244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 275/556] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:54.440316 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cea94-d42f-4cb9-8cc6-0af337f5d69b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.440605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cea94-d42f-4cb9-8cc6-0af337f5d69b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.440883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cea94-d42f-4cb9-8cc6-0af337f5d69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.449227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:54.452625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dac309-2cf4-4070-9c9b-6a9a5e33dd24 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.467327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3b1cea94-d42f-4cb9-8cc6-0af337f5d69b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:54.469065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3b1cea94-d42f-4cb9-8cc6-0af337f5d69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:54.497028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.497746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.498019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.498288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.509787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.511198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:54.527412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:54.528685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:54.529799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 282/557] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:54.530660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.555528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 276/558] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:54.556601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cea94-d42f-4cb9-8cc6-0af337f5d69b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.581814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:54.720854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7559a11-4925-406b-9f60-13919ebe5b56 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.733615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cea94-d42f-4cb9-8cc6-0af337f5d69b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.769799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.770347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.770619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.770892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.777686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.778601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.779084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.779833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.780231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.780984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:54.781471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.781706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.783564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:54.795759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:54.797220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 283/559] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:54.798146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.809262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 277/560] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:54.811317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.820437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:54.900200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:54.908115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32edaae-adb3-4117-a137-2b3b55b14428 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.925623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc695b6b-c769-4a99-9684-58f6eaf6583e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:54.940734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.941357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.941634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.941896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.952170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:54.957540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:54.958573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:54.959423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:54.963505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 284/561] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:54.964393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:54.968484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:54.969097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:54.969426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:54.969709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:54.970426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:54.970725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:54.972172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:54.996124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 278/562] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:54.996124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.058565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.086221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a13d23-8ac0-4dee-aef6-1c117751b4fc None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.098615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc124a-b808-4f67-bcf5-0d31fd8c90df None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.121253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.121953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.122253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.122522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.132836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.133469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.133741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.134017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.135887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.147012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:55.151610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:55.152550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:55.153608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 279/563] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:55.154433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.199570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.201117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.261497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aabb0-3ea7-4f23-9d23-860a993366e9 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.277410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.298940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.299523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.299809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.300077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.300765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:55.305427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.305573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 285/564] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:37:55.305573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.305725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.314455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 280/565] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:55.319575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.381608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.405697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a1936-1982-40c8-acbe-c38ee43eb416 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.425111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794daf5a-14d4-4ec4-927b-ba6dbae92ebf None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.438991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.439683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.439962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.440233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.449814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.451453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.456764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.457359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.457629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.457903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.458556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:55.458847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.460242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:55.465363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:55.466369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:55.467259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:55.467259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.467259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:55.467259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 286/566] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:55.467871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.468251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 281/567] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:55.469065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.502127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.571805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e745c0-0672-4c7c-8904-e5c6ceeaaf07 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.573907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaccbd5-3b98-48a2-9d87-b48792a85830 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.588254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.605207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104a302-c474-4ca9-b90c-f0b49ddfe644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.605830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104a302-c474-4ca9-b90c-f0b49ddfe644 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.606103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104a302-c474-4ca9-b90c-f0b49ddfe644 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.606407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104a302-c474-4ca9-b90c-f0b49ddfe644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.607675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.608350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.608622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.608957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.617638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104a302-c474-4ca9-b90c-f0b49ddfe644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.618954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.619064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104a302-c474-4ca9-b90c-f0b49ddfe644 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.620416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-0104a302-c474-4ca9-b90c-f0b49ddfe644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:55.633131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:55.634255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:55.635250 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:55.635250 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.635250 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:55.635423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:55.636364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:55.637563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:55.637563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.637563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:55.637825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 282/568] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:55.637915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 287/569] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:55.637915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.638051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662a767-865a-4b42-9269-1da3241f74b7 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.638051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662a767-865a-4b42-9269-1da3241f74b7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.638051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662a767-865a-4b42-9269-1da3241f74b7 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.638051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 283/570] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:55.638983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662a767-865a-4b42-9269-1da3241f74b7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.721488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.739382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c336b-770f-4205-8fc3-080c55801057 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.740034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662a767-865a-4b42-9269-1da3241f74b7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.771548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.772141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.772445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.772713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.773356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:55.773654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.774239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d50a5a-34fa-4036-99b8-527506e73334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.774953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d50a5a-34fa-4036-99b8-527506e73334 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.775047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.775230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d50a5a-34fa-4036-99b8-527506e73334 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.775491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d50a5a-34fa-4036-99b8-527506e73334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.785896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d50a5a-34fa-4036-99b8-527506e73334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.786654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 288/571] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:55.787308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d50a5a-34fa-4036-99b8-527506e73334 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.787429 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-37d50a5a-34fa-4036-99b8-527506e73334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:55.804283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:55.805318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:55.806192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:55.806192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:55.806192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:55.806192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 284/572] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:55.806782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af2f74-6202-4b96-ad69-40ec13905000 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.806948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af2f74-6202-4b96-ad69-40ec13905000 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.807298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af2f74-6202-4b96-ad69-40ec13905000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.825938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-76af2f74-6202-4b96-ad69-40ec13905000 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:55.826948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-76af2f74-6202-4b96-ad69-40ec13905000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:55.844248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:55.888149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af201dc4-d239-4d0b-9a0b-74d9c423f35d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:55.894803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 285/573] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:37:55.895636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af2f74-6202-4b96-ad69-40ec13905000 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.925371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:55.925975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:55.926249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:55.926514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:55.927149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:55.927453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:55.928871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:55.941929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 289/574] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:55.942761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:55.945418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.006593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af2f74-6202-4b96-ad69-40ec13905000 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.042324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.042984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.043257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.043521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.044879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:56.045318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.046810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.056714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92a42b-43e4-4fe2-8a19-45e28aa363cd None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.063086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 286/575] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:56.063086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.074811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.097170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.097876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.098241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.098621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.105392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.107014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:56.118242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:56.119299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:56.120425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:56.126336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 290/576] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:56.127243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.166248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f855015-1c34-4029-8880-735e17d28a3a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.199530 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.199713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.200390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.200681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.201021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.201780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:56.202056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.203446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.217362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 287/577] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:56.218171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.239916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446faadc-40b8-4b0b-a269-1de9138218a1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.273250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.273861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.274132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.274386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.280889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.282334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.287822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:56.294149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:56.295149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 291/578] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:56.300905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.317199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5fa0b-e14c-4c79-9ffb-751ff8685e77 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.351835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.352422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.352698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.352963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.360359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.361824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:56.373691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:56.374791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 288/579] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:56.375737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.401372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30313db6-0895-46dc-a379-e38bb9e41d9c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.427291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.439255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.440074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.440368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.440637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.441326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:56.441630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.443050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.454604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 292/580] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:56.455388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.483973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a441d-c7a2-443f-a9f5-12cec97eeab4 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.514063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.520057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.520704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.520976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.521241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.532311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:56.538767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:56.539846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 289/581] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:56.540715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.562523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5357e64-f440-4533-a3d9-914ca5635b01 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.596653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.597264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.597532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.597791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.604632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.606012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:56.617185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:56.618297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 293/582] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:56.619304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.642659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2a8ee-68bc-4744-a473-c697857de169 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.660354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.677407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.678032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.678298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.678560 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.679225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:56.679520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.680988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.692678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 290/583] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:56.693535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.716059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bece0dc-86fd-4ddd-b2c0-26043cb67d1e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.747646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.749069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.749766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.750087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.750356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.751079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:56.751375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.752869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.769923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 294/584] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:56.770708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.793912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555717e-d1ff-44b0-8750-96910548a14c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.833384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.834076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.834298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.834564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a25194d34d67429b92944e0484de03cd/roles/db7026479a164eec9e413091ad5440fe` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.850051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=a25194d34d67429b92944e0484de03cd, role_id=db7026479a164eec9e413091ad5440fe)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.851489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.864436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 291/585] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:56 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a25194d34d67429b92944e0484de03cd/roles/db7026479a164eec9e413091ad5440fe => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:56.865178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.869502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:56.873260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc2d0b-2aac-4060-b17c-ab20abbe701d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.914787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:56.915397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:56.915668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:56.915929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:56.916643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:56.916965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:56.918397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:56.932133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 295/586] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:56.932133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:56.988580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c28ce5-ad49-4a74-89fe-e765714bb1bc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:56.990319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:57.035026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.035696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.036531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.036807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.037542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:57.037842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.039523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.049075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb7fec-93c9-41e8-8dd2-2ca7107fe854 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.057599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 292/587] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:57.058346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.087516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.088501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.088900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.089298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.090215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=01afe498af8c42e8bca4e993897ad1d9) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:57.090742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.092425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.098002 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 296/588] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:37:56 2022] GET /identity/v3/role_assignments?user.id=01afe498af8c42e8bca4e993897ad1d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:57.099425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.149175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:57.184203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d867-b9a3-4f30-bd18-d212f984b566 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.222475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.223111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.223382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.223645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.237849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.239645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.247593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1871-5459-47d5-9b02-345afc2e785f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:57.259015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:57.260219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:57.261714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:57.261714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:57.261714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:57.261714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 293/589] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:57.263001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.283730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:57.301983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.303407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.303881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.304365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.317957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.320361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:57.341124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:57.342110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:57.342983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:57.342983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:57.342983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:57.345427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 297/590] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:57.346407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.382074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f562f93-78a7-4d2c-93de-134f24dead15 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.426924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.428694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.432543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.432950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.437818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:57.450351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.452891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.461580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8767c-cd36-4775-b89d-bee62a137ab8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:57.472057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:57.473211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:57.474297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:57.474297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:57.474297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:57.476673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 294/591] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:57.477816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.508754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.509508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.509880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.510246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.511008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:57.511403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.514009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.526786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 298/592] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:57.532412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.559156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:57.609024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f18fd2-8b77-4057-9ed1-06cadec704d5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.642118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2d57-4bac-496a-8711-748a0aca5639 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.657667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.658844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.659769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.660238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.661340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:57.662094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.664074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.679801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.680371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.680653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.680919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.685859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:57.685859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.685859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.686639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 295/593] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:57.688249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.692565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:57.704734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 299/594] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:57.704734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.811325 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53030f-c62d-42ba-a551-e877d2cab172 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.824591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3816d16-1ae8-4fc1-8906-4b1601e0e9e1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:57.854755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.856407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:57.857655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.858152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.858657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.870658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:57.870836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.871701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:57.872238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:57.872390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.872855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:57.881163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:57.882836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:37:57.885151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:57.886075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 10:37:57.887321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:57.891887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 300/595] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:57.892764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:57.902492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:57.903614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:57.904544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:57.907746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 296/596] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:57.909023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:57.971078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:58.027515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13645e1-b78e-43a6-ba6a-26177217a030 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.048954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d29b43-5c11-4918-9ac3-72dcff015712 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.073732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.074685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.075213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.075719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.085784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.087890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.096991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.097596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.097867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.098130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.098853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:58.099152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:58.100481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:58.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:58.102713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:58.103141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.104512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 301/597] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:58.105423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:58.123947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 297/598] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Aug 12 10:37:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:58.125191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:58.181373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:58.211852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253b5a7-f1b0-4a27-b628-98f246554f5c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.237167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:58.256541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0de706-145b-4191-8183-8d36626dc238 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.260698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.261382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.261724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.262359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.263598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:58.263711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.265224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.281070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 302/599] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:58.282298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:58.296916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.297794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.298329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.298791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.299730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:58.300241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.301889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.314963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 298/600] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:58.316058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:58.421926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbcf3f-aa85-4085-a3c8-e1b595a33f3d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.427762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f21436-e517-400e-a3d0-4b1837790ca2 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.429692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:58.466647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.467445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.467949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.468065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.468347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.468611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.468959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.469414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/76f4fb448f1443b7a77fd8e1954defe5/roles/db7026479a164eec9e413091ad5440fe` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.481110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.482973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.486485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=76f4fb448f1443b7a77fd8e1954defe5, role_id=db7026479a164eec9e413091ad5440fe)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.488283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:58.501555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:58.502517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:58.503436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:58.503436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:58.503436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:58.503436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 303/601] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:58.504115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:58.508744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 299/602] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:58 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/76f4fb448f1443b7a77fd8e1954defe5/roles/db7026479a164eec9e413091ad5440fe => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:58.509182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:58.539545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:58.618399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e566f-2b79-4c45-b7f2-b4c8dcf33feb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.657090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.657687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.657956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.658226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/8571f3d80b0749729012353fcbbd9e4b/roles/db7026479a164eec9e413091ad5440fe` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.660346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4f771-2112-4a95-8f4c-a2459df66685 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.678246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=8571f3d80b0749729012353fcbbd9e4b, role_id=db7026479a164eec9e413091ad5440fe)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.679695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.696777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 304/603] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:58 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/8571f3d80b0749729012353fcbbd9e4b/roles/db7026479a164eec9e413091ad5440fe => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:58.707578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7bce1-caf4-4de6-8718-6c6976c824f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.708656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7bce1-caf4-4de6-8718-6c6976c824f2 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.709210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7bce1-caf4-4de6-8718-6c6976c824f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.709695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7bce1-caf4-4de6-8718-6c6976c824f2 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.710582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7bce1-caf4-4de6-8718-6c6976c824f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:58.711094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7bce1-caf4-4de6-8718-6c6976c824f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.712953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7bce1-caf4-4de6-8718-6c6976c824f2 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.727556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 300/604] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:58.735902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:58.795038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28904e96-d7c7-4234-bd9f-61acdac5a208 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.795645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28904e96-d7c7-4234-bd9f-61acdac5a208 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.796113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28904e96-d7c7-4234-bd9f-61acdac5a208 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.797848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 301/605] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:37:58.803835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf98da9-bf6a-44d0-b0fe-cf2be2e8ab24 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.804155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf98da9-bf6a-44d0-b0fe-cf2be2e8ab24 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.804437 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf98da9-bf6a-44d0-b0fe-cf2be2e8ab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.828877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-baf98da9-bf6a-44d0-b0fe-cf2be2e8ab24 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:58.830114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-baf98da9-bf6a-44d0-b0fe-cf2be2e8ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:58.842267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:58.845182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:58.870938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ead43-5206-4a7a-b88d-fa854a5ee481 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:58.913076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:58.913695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:58.913964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:58.914227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:58.921698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:58.923233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:58.932350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 302/606] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:37:58.938600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:58.939496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:58.940459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 305/607] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:58.945727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf98da9-bf6a-44d0-b0fe-cf2be2e8ab24 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.056364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf98da9-bf6a-44d0-b0fe-cf2be2e8ab24 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:59.093947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712750f-982e-49fc-93e0-02d1049655d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:59.094554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712750f-982e-49fc-93e0-02d1049655d7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.094827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712750f-982e-49fc-93e0-02d1049655d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.095093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712750f-982e-49fc-93e0-02d1049655d7 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.095787 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712750f-982e-49fc-93e0-02d1049655d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:59.096092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712750f-982e-49fc-93e0-02d1049655d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:59.097605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712750f-982e-49fc-93e0-02d1049655d7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:59.115892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 303/608] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:37:59.121626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.261240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78476a43-3286-42fc-8b30-e1338205a7c3 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.261433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b1ef-3947-4dfd-93b0-ec8e4d65b49c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:59.262311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78476a43-3286-42fc-8b30-e1338205a7c3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.263045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78476a43-3286-42fc-8b30-e1338205a7c3 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.267568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 304/609] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:37:59.270018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0903cb6-3e41-44bb-9aaf-2d015fa7dbfc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.270400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0903cb6-3e41-44bb-9aaf-2d015fa7dbfc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.270767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0903cb6-3e41-44bb-9aaf-2d015fa7dbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.277216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:59.290016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a0903cb6-3e41-44bb-9aaf-2d015fa7dbfc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:37:59.291605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a0903cb6-3e41-44bb-9aaf-2d015fa7dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:37:59.311936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:59.312860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.313298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.313772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a8f3a56366d348bfb0a5c445905dc46b/roles/db7026479a164eec9e413091ad5440fe` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.329791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:59.334043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=a8f3a56366d348bfb0a5c445905dc46b, role_id=db7026479a164eec9e413091ad5440fe)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:59.335712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:59.358506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 306/610] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:37:59 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a8f3a56366d348bfb0a5c445905dc46b/roles/db7026479a164eec9e413091ad5440fe => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:37:59.359100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.374744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 305/611] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:37:59.477264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1f1db-79b7-4194-9a97-8fc4d4ed1db8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:59.516572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:59.517598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.518069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.518578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.530490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:59.535583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:59.536550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:59.537400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:59.540036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 307/612] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:59.546296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0903cb6-3e41-44bb-9aaf-2d015fa7dbfc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.654856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.656636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0903cb6-3e41-44bb-9aaf-2d015fa7dbfc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:59.691953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:59.697548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:59.698500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.698972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.699438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.701790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:37:59.702320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:59.704039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:59.713046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 306/613] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:37:59.716391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:37:59.719773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.773559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d8b33-48f0-461f-bf96-f06fb0e8e694 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:59.810546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:59.811420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.811835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.812240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.823990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:37:59.825871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e9acf-61a3-48a6-9687-0a3279f9279b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:37:59.829912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:59.831062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:37:59.832081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:37:59.834382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 308/614] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:59.840246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.871490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:37:59.872191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:37:59.872790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:37:59.873080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:37:59.894917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:37:59.896477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:37:59.929943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:37:59.930931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:37:59.931855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:37:59.931855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:37:59.931855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:37:59.933332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 307/615] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:37:59.938910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:37:59.989885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419b5bf-c88b-44df-81fb-0b9710124fcf None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.000841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:00.034780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:00.041579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.042877 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.043150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.043414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.044392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:00.044650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.046138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.064017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 309/616] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:37:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:00.069557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:00.094118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328a33-1acc-43a4-bc8b-09a98a434bcb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.130323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.131299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.131776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.132293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.133368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:00.133902 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.135616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.149390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 308/617] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:00.155324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:00.210014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355012d-a477-4339-a8b5-e8bca7885077 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.246403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.247072 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.247367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.247870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.260755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.262778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.265266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c846c6-43c9-40f0-a3a1-8d81ead86dc3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.281408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:00.287019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:00.287984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:00.289293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:00.289293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:00.289293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:00.289293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 310/618] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:00.294940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:00.307909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.308869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.309447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.309938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] PATH_INFO: `/v3/users/nova` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.317972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.319713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:00.332638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:00.333738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:00.334842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:00.338111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 309/619] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:00.343407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:00.345522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:00.451939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eacb54-0cb7-4d9d-8650-ca28f519bebc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.487433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.488472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.488983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.489496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.490420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:00.490937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.492619 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.494390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0bf0a-da4f-4a96-9b2e-564900816c2c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.511744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 310/620] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:00.518201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:00.553938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:00.585055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.585609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.585881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.586160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.586820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:00.587170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.588558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.622790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 311/621] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:00.625935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7cfe8-522d-4369-ac5b-723545949d3f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.633253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:00.663468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.664594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.665095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.665575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.737850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.739772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.836630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:00.848727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 311/622] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:38:00.850537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:00.886086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:00.907028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a0653-a9c2-4b07-90f9-64ea245fd15c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:00.943314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c8241-dc0c-4bb9-bf65-b864c60cf078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:00.943941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c8241-dc0c-4bb9-bf65-b864c60cf078 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.944213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c8241-dc0c-4bb9-bf65-b864c60cf078 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.944510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c8241-dc0c-4bb9-bf65-b864c60cf078 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.952094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c8241-dc0c-4bb9-bf65-b864c60cf078 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:00.953606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c8241-dc0c-4bb9-bf65-b864c60cf078 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-490c8241-dc0c-4bb9-bf65-b864c60cf078 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:00.965338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:00.966339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:00.967283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 312/623] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:00.968408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f914d-c002-46f7-8198-9935371be4a4 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:00.968601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f914d-c002-46f7-8198-9935371be4a4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:00.968874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f914d-c002-46f7-8198-9935371be4a4 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:00.974331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 313/624] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:00.974331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f914d-c002-46f7-8198-9935371be4a4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.018464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa4c84-f600-4322-89f5-7194e1f949f1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.050418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.067357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976c22b-6221-437a-ab04-2ccf236a8b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.068960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976c22b-6221-437a-ab04-2ccf236a8b13 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.069433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976c22b-6221-437a-ab04-2ccf236a8b13 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.070117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976c22b-6221-437a-ab04-2ccf236a8b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.084964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976c22b-6221-437a-ab04-2ccf236a8b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.087205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976c22b-6221-437a-ab04-2ccf236a8b13 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.088281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f914d-c002-46f7-8198-9935371be4a4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-3976c22b-6221-437a-ab04-2ccf236a8b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:01.108731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:01.109748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:01.110624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:01.110624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:01.110624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:01.110624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 312/625] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:01.111541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7195b-5b60-4850-8ecb-6e62e0e8e9ce None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.111827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7195b-5b60-4850-8ecb-6e62e0e8e9ce None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.112101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7195b-5b60-4850-8ecb-6e62e0e8e9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.130441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.131050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.131158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-67a7195b-5b60-4850-8ecb-6e62e0e8e9ce None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:01.131317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.131586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.132355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:01.132459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-67a7195b-5b60-4850-8ecb-6e62e0e8e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:01.132649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.134086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.153651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 314/626] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Aug 12 10:38:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:01.154682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.157381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.205955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 313/627] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:01.207218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7195b-5b60-4850-8ecb-6e62e0e8e9ce None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.286642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be69d-5681-4c57-a3b2-c0d0f344f4eb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.312930 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7195b-5b60-4850-8ecb-6e62e0e8e9ce None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.321686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.338511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92820fae-3c1c-43c2-91a3-eecc1a2c57de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.339138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92820fae-3c1c-43c2-91a3-eecc1a2c57de None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.339413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92820fae-3c1c-43c2-91a3-eecc1a2c57de None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.339682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92820fae-3c1c-43c2-91a3-eecc1a2c57de None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.340864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92820fae-3c1c-43c2-91a3-eecc1a2c57de None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:01.341246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92820fae-3c1c-43c2-91a3-eecc1a2c57de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.343233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92820fae-3c1c-43c2-91a3-eecc1a2c57de None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.356223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879b02c-928a-4cf6-9d79-34e2aa250530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.357288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b02c-928a-4cf6-9d79-34e2aa250530 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.357768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b02c-928a-4cf6-9d79-34e2aa250530 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.357874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 315/628] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:01.358425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b02c-928a-4cf6-9d79-34e2aa250530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.359060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d3e7-e845-4d32-8a27-23e72d9deb3b None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.364028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d3e7-e845-4d32-8a27-23e72d9deb3b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.364394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d3e7-e845-4d32-8a27-23e72d9deb3b None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.366175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 316/629] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:01.367357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d3e7-e845-4d32-8a27-23e72d9deb3b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.376245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879b02c-928a-4cf6-9d79-34e2aa250530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.378159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879b02c-928a-4cf6-9d79-34e2aa250530 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2879b02c-928a-4cf6-9d79-34e2aa250530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:01.397714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:01.398819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:01.400332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:01.400332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:01.400332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:01.400332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 314/630] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:01.400332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc513a-9417-44b2-a907-36a40b19dfc4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.400636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc513a-9417-44b2-a907-36a40b19dfc4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.400883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc513a-9417-44b2-a907-36a40b19dfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.423647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-49bc513a-9417-44b2-a907-36a40b19dfc4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:01.425026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-49bc513a-9417-44b2-a907-36a40b19dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:01.437547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.483088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d3e7-e845-4d32-8a27-23e72d9deb3b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.514156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 315/631] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:01.515116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc513a-9417-44b2-a907-36a40b19dfc4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.525199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.526574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.527052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.527891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.537092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.539574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:01.555039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:01.555999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:01.556945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:01.559226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 317/632] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:01.560163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.623044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.640932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc513a-9417-44b2-a907-36a40b19dfc4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.673628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f489f2-4839-42ea-8ad8-0d4e4e883b5c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.687410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.688748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.689196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.689846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.690649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:01.691310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.693253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.698245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.711261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 316/633] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:01.712488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.713059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.713890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.714257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.714621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.726497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:01.732301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:01.733456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:01.734510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:01.735855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 318/634] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:01.736761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.834255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2314f3-ac09-4a6a-b5d2-7a2978d09095 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.852568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3125716-8b5c-4e34-b99b-39677e664dc7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:01.866869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.867472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.867743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.868009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.868723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:01.869013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.870429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.882108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 317/635] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:01.883273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.888891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:01.889539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:01.889798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:01.890069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:01.893843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.897454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:01.898911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:01.910173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:01.911171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 319/636] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:01.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:01.958337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:01.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8023486-4c65-44f8-9c46-8436c7bb275a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.014735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5748f-9a3b-47a8-82ce-1dd6e264993c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.016101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe9f24-37dd-417e-8d1f-199da832c4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.016740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe9f24-37dd-417e-8d1f-199da832c4c7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.017018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe9f24-37dd-417e-8d1f-199da832c4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.017284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe9f24-37dd-417e-8d1f-199da832c4c7 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.017941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe9f24-37dd-417e-8d1f-199da832c4c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:02.018238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe9f24-37dd-417e-8d1f-199da832c4c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.019663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe9f24-37dd-417e-8d1f-199da832c4c7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.027480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 318/637] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:02.028715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e08917-6351-4ebb-9a64-60e7347ea2c0 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.028994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e08917-6351-4ebb-9a64-60e7347ea2c0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.029264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e08917-6351-4ebb-9a64-60e7347ea2c0 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.030451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 319/638] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:02.031133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e08917-6351-4ebb-9a64-60e7347ea2c0 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.054124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.054749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.055023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.055290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.065273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:02.070339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:02.071462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 320/639] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:02.072343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.102883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:02.135845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e08917-6351-4ebb-9a64-60e7347ea2c0 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.169132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6a85c-9e94-4134-9819-7b6e7c8a35b9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.170633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:02.171196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.171823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.172094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.172397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.173137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a25194d34d67429b92944e0484de03cd) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:02.173438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.174922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.181058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 320/640] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/role_assignments?user.id=a25194d34d67429b92944e0484de03cd&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:02.181878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.220179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5528222-fc6d-416d-b025-1487693919ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.220838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528222-fc6d-416d-b025-1487693919ec None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.221103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528222-fc6d-416d-b025-1487693919ec None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.221404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528222-fc6d-416d-b025-1487693919ec None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.222096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5528222-fc6d-416d-b025-1487693919ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:02.223083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5528222-fc6d-416d-b025-1487693919ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.223900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5528222-fc6d-416d-b025-1487693919ec None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.243676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 321/641] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:02.243676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f78dd-672d-434f-a1e5-5738e0db5291 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.243676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f78dd-672d-434f-a1e5-5738e0db5291 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.243676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f78dd-672d-434f-a1e5-5738e0db5291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.261564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-313f78dd-672d-434f-a1e5-5738e0db5291 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:02.263246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-313f78dd-672d-434f-a1e5-5738e0db5291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:02.306972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac3cf6-7672-41f4-a965-7a81e7942d16 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.343354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 322/642] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:02.343354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f78dd-672d-434f-a1e5-5738e0db5291 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.348227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.348939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.349215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.349481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.351446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:02.361251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.362716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:02.379611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:02.380744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:02.381660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:02.381660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:02.381660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:02.381660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 321/643] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:02.381660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.460694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:02.471825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f78dd-672d-434f-a1e5-5738e0db5291 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.511054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e313e-67cd-48c3-b188-cbe1c07c6761 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.512090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.513079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.513462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.513734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.514404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:02.514702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.516087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.528736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 323/644] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:02.529555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.559984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.560971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.561250 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.561920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/01afe498af8c42e8bca4e993897ad1d9/roles/db7026479a164eec9e413091ad5440fe` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.582346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=01afe498af8c42e8bca4e993897ad1d9, role_id=db7026479a164eec9e413091ad5440fe)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.584343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.604317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 322/645] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:38:02 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/01afe498af8c42e8bca4e993897ad1d9/roles/db7026479a164eec9e413091ad5440fe => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:38:02.605169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.637612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaea94-0a20-490d-bed4-670c5398be91 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.657761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:02.697747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.698355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.698626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.698873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.701309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:02.701573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.703241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.720686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 324/646] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:02.721899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.743670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:02.758018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70119ce-1aa5-48fb-ae9d-c8bd9894e00a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.805287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.807104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.807104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.807104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.821528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:02.821528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:02.839729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:02.840932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:02.842004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:02.842004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:02.842004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:02.842004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 323/647] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:02.842004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.854975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9c50b-62a4-45ea-82bd-d97d58565da6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:02.890917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:02.891457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:02.891729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:02.891996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:02.902095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:02.906983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:02.907926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:02.908808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 325/648] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:02.909505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:02.946786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:02.960699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c80d06-48ca-41b6-9879-63fc2eef82eb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.000311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.000950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.001222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.001489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.002936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.010251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.011699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:03.025487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:03.026923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 324/649] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:03.027876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.045355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc8d7-aca6-47bc-8d50-6358f2e3bd9a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.087792 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.088433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.088704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.088970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.089607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:03.089901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.091534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.103527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 326/650] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:03.104481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.152331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fe752-8ad2-497f-8ba3-46326bc03e15 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.191133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.191791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.192066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.192371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.193032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:03.193328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.194710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.208238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 325/651] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:03.209172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.216984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801aa9c-d466-4af6-823f-50be1c785cab None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.222461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.263789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.264414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.264684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.264950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.266178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:03.266485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.268173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.270789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.289888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 327/652] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:03.290756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.324242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6073475-84d9-4629-89dc-de4f04e281cd None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.368109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.369501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.370196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.370639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.381558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.383815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:03.400878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:03.401857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 12 10:38:03.402888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:03.410040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 326/653] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:03.410040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.425882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fffd-d130-4efe-a02a-5a018cede1a0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.469367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.469960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.470588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.470854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.483534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.485547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.497031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:03.503200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:03.504162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:03.505649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:03.505649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:03.505649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:03.505649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 328/654] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:03.505649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.540515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14df82-daea-4f44-9b87-c36bdeb5a097 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.563020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.583154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.583759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.584026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.584303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.625798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704edba6-4c9c-4366-aeeb-6fbf9d2fcaa9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.654023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.655418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.663451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.664458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.664932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.665397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.666479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:03.666979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.668732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.692903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 329/655] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:03.694179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.725491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.764100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 327/656] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:38:03.765292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.801523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313ee5c-8909-4c6f-949a-352fc14059ee None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.803405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.837783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.838789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.839264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.839725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.840673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:03.841297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.842881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:03.855013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 330/657] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:03.856253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:03.869089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f98c5-5e3b-463f-9a2c-f5efb831ede1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.904425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:03.904949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:03.905221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:03.905543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:03.948069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:03.966813 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f4427-343f-462d-a684-561a0a36dbac None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:03.983894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:03.985560 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.010106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.011155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.011576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.012729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.027645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.029849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:04.048515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:04.049739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:04.050860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:04.050860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:04.050860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:04.053221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 331/658] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:04.054238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.077239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:04.116336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 328/659] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:38:04.117821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.173826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc17cdf-8220-4436-9c38-3fa0ce06becd None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.188672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:04.212293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.213142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.213511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.213871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.222524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487f4b2-85ac-4e52-af54-b6c7bc79eb25 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.222935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.224512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:04.237131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:04.238207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:04.239327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 332/660] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:04.240110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.261646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.262995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.263469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.263924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.264784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:04.265265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.266810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.279330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 329/661] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:04.280491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.328795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:04.363394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cda3e-6fa4-40d4-80ff-dc839de96f75 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.393231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24103bb3-10e7-4905-aa31-52739a0de478 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.398735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.399379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.399651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.399915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.411736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.413217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.422397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:04.428387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb6fa2-7098-4da9-86cf-3be79495900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.428935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb6fa2-7098-4da9-86cf-3be79495900f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.429208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb6fa2-7098-4da9-86cf-3be79495900f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:04.429317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:04.435603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:04.436592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:04.436592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:04.436592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:04.436767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb6fa2-7098-4da9-86cf-3be79495900f None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.436767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb6fa2-7098-4da9-86cf-3be79495900f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:04.436893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 333/662] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:04.436960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb6fa2-7098-4da9-86cf-3be79495900f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.437036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.437105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb6fa2-7098-4da9-86cf-3be79495900f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.455807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 330/663] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:04.456900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1081252-4cd7-4406-881d-f39d751bd4ff None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.457179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1081252-4cd7-4406-881d-f39d751bd4ff None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.457446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1081252-4cd7-4406-881d-f39d751bd4ff None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.458873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 331/664] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:04.459569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1081252-4cd7-4406-881d-f39d751bd4ff None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.543345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c14028-fb8b-40f5-9895-c86c5af521bf None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.566463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1081252-4cd7-4406-881d-f39d751bd4ff None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.571746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:04.582198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.582817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.583083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.583349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.591118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.592596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.603889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb482d-6a2e-4e6f-801e-33b53e3b1377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:04.604058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:04.605079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:04.606322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:04.606851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb482d-6a2e-4e6f-801e-33b53e3b1377 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.606851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb482d-6a2e-4e6f-801e-33b53e3b1377 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.606851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb482d-6a2e-4e6f-801e-33b53e3b1377 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.607078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 334/665] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:04.607078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.610689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb482d-6a2e-4e6f-801e-33b53e3b1377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:04.610998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb482d-6a2e-4e6f-801e-33b53e3b1377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.612408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb482d-6a2e-4e6f-801e-33b53e3b1377 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.624114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 332/666] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:04.625441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5cfe-ca8e-43ae-9c22-d2fb331820e6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.625727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5cfe-ca8e-43ae-9c22-d2fb331820e6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.626000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5cfe-ca8e-43ae-9c22-d2fb331820e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.644210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d26b5cfe-ca8e-43ae-9c22-d2fb331820e6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:04.645421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d26b5cfe-ca8e-43ae-9c22-d2fb331820e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:04.689313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:04.707899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74af68-51ec-4244-b9fc-3148c649b0c2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.712701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 333/667] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:04.713983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43123188-cfad-4d08-bae0-bc1f8631bfa2 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.714262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43123188-cfad-4d08-bae0-bc1f8631bfa2 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.714530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43123188-cfad-4d08-bae0-bc1f8631bfa2 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.716385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 334/668] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:04.716566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43123188-cfad-4d08-bae0-bc1f8631bfa2 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.741962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.742565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.742950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.743220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.800629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:04.814463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.815929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.823381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43123188-cfad-4d08-bae0-bc1f8631bfa2 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:04.922174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 335/669] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:38:04.923329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:04.981731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b823dca-915a-4a68-b280-7ff58a562c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:04.982249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823dca-915a-4a68-b280-7ff58a562c1e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.982967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823dca-915a-4a68-b280-7ff58a562c1e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:04.983082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823dca-915a-4a68-b280-7ff58a562c1e None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:04.983883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b823dca-915a-4a68-b280-7ff58a562c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:04.984172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b823dca-915a-4a68-b280-7ff58a562c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:04.985632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b823dca-915a-4a68-b280-7ff58a562c1e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:04.998188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 335/670] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:04.999531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6e4a2-0693-4ca6-a263-946fd5a06c4d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:04.999812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6e4a2-0693-4ca6-a263-946fd5a06c4d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.000077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6e4a2-0693-4ca6-a263-946fd5a06c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.014327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.018018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5cf6e4a2-0693-4ca6-a263-946fd5a06c4d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:05.019209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5cf6e4a2-0693-4ca6-a263-946fd5a06c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:05.029940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3d269-991a-4f6d-867d-0c16ae3309f5 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.048022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.063053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.063666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.063935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.064193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.070873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.072335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:05.083712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:05.089414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:05.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 336/671] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:05.091414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 336/672] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:05.091501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.091609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6e4a2-0693-4ca6-a263-946fd5a06c4d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.195557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6e4a2-0693-4ca6-a263-946fd5a06c4d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.196238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551da201-2540-40db-a7cf-c783e3a5d33b None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.234080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.234637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.234912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.235180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.240639 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.241256 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.241527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.241789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.246137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:05.251060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:05.252003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:05.253228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 337/673] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:05.254025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.257955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.259447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.267581 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:05.276787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:05.277844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:05.283451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:05.283451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:05.283451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:05.283451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 337/674] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:05.283451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.291113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.373351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f89bbe-643f-45c8-b933-0574e51a254d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.384879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b463dd-83c9-4c34-bfc5-597aae113a9a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.406518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.407111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.407384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.407646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.408379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=76f4fb448f1443b7a77fd8e1954defe5) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:05.408684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.410116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.415979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 338/675] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/role_assignments?user.id=76f4fb448f1443b7a77fd8e1954defe5&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:05.416849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.419344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.419957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.420235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.420549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.421251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:05.421555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.423038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.436751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 338/676] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:05.437522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.505970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.533931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.544368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffbc2-348b-44f6-aca7-99c0eb57ce33 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.556275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a9bfc-8582-48e1-bf5f-bc7116c01af2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.583682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.584304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.584577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.584839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.585496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:05.585789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.587146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.595528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.596135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.596275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 339/677] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:05.596544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.596818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.596995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.597563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:05.597869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.599344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.612696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 339/678] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:05.613445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.709948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc311c58-1245-476e-9110-54320f7929d1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.714485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dd341-ef8d-4e16-ae78-d4021d91a3a4 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.731797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.746419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.747228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.747559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.747753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.750827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.751432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.751707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.751975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.752729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=8571f3d80b0749729012353fcbbd9e4b) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:05.753036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.754487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.759261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:05.760784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 340/679] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/role_assignments?user.id=8571f3d80b0749729012353fcbbd9e4b&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:05.761498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:05.781197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:05.782215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 340/680] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:05.783063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:05.812464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:05.911179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19174b-0b06-4e30-baec-ede442924480 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.913282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863c79e-d22a-464f-a623-4acad1ec962e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:05.959088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.959324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:05.959847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.960121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.960220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:05.960412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.960505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:05.960770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:05.970497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.972158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.973268 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:05.974716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:05.986714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:05.987655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:05.988578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 341/681] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:05.989060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:06.001808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:06.002800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:06.003764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:06.003764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:06.003764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:06.003764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 341/682] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:06.003764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.051552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:06.095702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:06.117563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86f16c-f044-4b9f-b5c1-d4d08c8097e0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.140960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feecba9-5cd2-418d-b5c6-723b6f3fb17c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.152223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.152968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.153388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.153388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.154154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:06.154451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.155851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.168320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 342/683] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:06.169996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.196659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.198290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.198766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.199424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.201350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:06.202201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.204358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.221772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 342/684] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:06.223314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.292763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d232e43-3f49-411d-9e86-ada10bf52943 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.329318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.329967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.330338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.330691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.333007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:06.333007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.333526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.338649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:06.358312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 343/685] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:06.359457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.365173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:06.374723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aeb2e-f797-4ccc-967e-c39ad7bfa0ba None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.424378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.425755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.426173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.426578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.440550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.443494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:06.464597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:06.465820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:06.466741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:06.466741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:06.466741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:06.469241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 343/686] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:06.470246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.489921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6529e-37b5-4153-bdf8-12e85c3bdf9b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.527124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.528235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.528859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.529274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.530196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a8f3a56366d348bfb0a5c445905dc46b) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:06.530636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.532400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.539903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 344/687] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/role_assignments?user.id=a8f3a56366d348bfb0a5c445905dc46b&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:06.541004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.581007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f98ae-a9b1-4336-bfdc-8153020e65ef None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.605620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:06.630475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.631186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.631465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.631734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.651414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.654440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:06.673421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:06.674436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:06.675352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:06.678472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 344/688] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:06.680098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.692639 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:06.706854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57098b8b-82fa-4a64-a396-71bdcf1d55ad None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.762738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.763605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.763962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.764350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.765054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:06.765410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.766846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.780189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 345/689] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:06.787460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.834269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50886c6-a98e-45a8-b71d-31bc2eff57c6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.871150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.872173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.872780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.873236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.874157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:06.874648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.876676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.898015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 345/690] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:06.903825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.906555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d99640-26ec-4ab5-b75a-71e78ef5eaad None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:06.931399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:06.947602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:06.948252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:06.948552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:06.948817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:06.957145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:06.958647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:06.971840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:06.972921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:06.973868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:06.975190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 346/691] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:06.980607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:06.985152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:07.034550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edf1cd-670e-491c-a58c-2cff618a08a2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:07.078049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:07.078656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.078926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.079190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.098529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e47e37-bb6f-4ac0-8c66-3424aeddf2f7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:07.154302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:07.155771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:07.158410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:07.159028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.159300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.159564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.160329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:07.160637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:07.162075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:07.206539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 347/692] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:07.216104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:07.236732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:07.270456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 346/693] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:38:07.276181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:07.328867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:07.403818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d18a5-0299-4aaa-836e-8aa07168097c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:07.439216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:07.439841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.440110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.440405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.451277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:07.452758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:07.469060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:07.470087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:07.471161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:07.471161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:07.471161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:07.471161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 347/694] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:07.475862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:07.502314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:07.508698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4ad11-8cd4-41e7-8bc2-b84d480c918a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:07.598622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13648d1-65ae-497d-9888-d74575d404eb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:07.599989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a22c9-5a73-4fd5-af67-0984c22d01cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:07.600819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a22c9-5a73-4fd5-af67-0984c22d01cd None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.600987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a22c9-5a73-4fd5-af67-0984c22d01cd None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.601342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a22c9-5a73-4fd5-af67-0984c22d01cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.638218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20d7ec-462a-41f9-973a-c4af9e4c5a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:07.639131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20d7ec-462a-41f9-973a-c4af9e4c5a73 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.639573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20d7ec-462a-41f9-973a-c4af9e4c5a73 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.639985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20d7ec-462a-41f9-973a-c4af9e4c5a73 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.640805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20d7ec-462a-41f9-973a-c4af9e4c5a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:07.640805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20d7ec-462a-41f9-973a-c4af9e4c5a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:07.642469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20d7ec-462a-41f9-973a-c4af9e4c5a73 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:07.656381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 348/695] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:07.657494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269dc3-4aed-42f9-9552-e90e6cf56aa5 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.657783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269dc3-4aed-42f9-9552-e90e6cf56aa5 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.658061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f269dc3-4aed-42f9-9552-e90e6cf56aa5 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.660453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 349/696] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:07.665640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269dc3-4aed-42f9-9552-e90e6cf56aa5 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:07.693980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a22c9-5a73-4fd5-af67-0984c22d01cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:07.696870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a22c9-5a73-4fd5-af67-0984c22d01cd None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:07.747761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:07.762244 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:07.794543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f269dc3-4aed-42f9-9552-e90e6cf56aa5 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:07.835611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 348/697] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 10:38:07.836734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:07.837051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6961a-eb6e-4678-98d4-3b36080bd38f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.837334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6961a-eb6e-4678-98d4-3b36080bd38f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.837507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:07.837630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6961a-eb6e-4678-98d4-3b36080bd38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.837748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:07.838098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:07.846080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:07.848180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:07.860570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b8b6961a-eb6e-4678-98d4-3b36080bd38f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:07.861225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:07.862086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:07.862986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:07.863420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b8b6961a-eb6e-4678-98d4-3b36080bd38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:07.867282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 350/698] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:07.868193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:07.943597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 349/699] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:07.944632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6961a-eb6e-4678-98d4-3b36080bd38f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:07.973443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a1e1d-4922-4558-8753-1b2105c1f34c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.008463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.009087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.009407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.009795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.015478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:08.022262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.023985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:08.040096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:08.042322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:08.044376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:08.044376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:08.044376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:08.044376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 351/700] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:08.048281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:08.055408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.057280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6961a-eb6e-4678-98d4-3b36080bd38f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.094631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.095241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.095517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.095789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.096517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:08.096820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.098346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.116501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 350/701] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:08.118523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.237345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674bd9-9b84-4399-a0f0-1247114dc779 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.258469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cbdf7-48c0-4e9b-af84-3e564406a93c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.272798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9c22f-26d6-44c6-a3ec-432980b896ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.273705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9c22f-26d6-44c6-a3ec-432980b896ba None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.274170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9c22f-26d6-44c6-a3ec-432980b896ba None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.274685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9c22f-26d6-44c6-a3ec-432980b896ba None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.275522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9c22f-26d6-44c6-a3ec-432980b896ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:08.276009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9c22f-26d6-44c6-a3ec-432980b896ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.277596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9c22f-26d6-44c6-a3ec-432980b896ba None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.287502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:08.290452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 352/702] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:08.293721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2a9c-6ef5-47bc-891e-ac6d30e03694 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.293721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2a9c-6ef5-47bc-891e-ac6d30e03694 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.293721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f2a9c-6ef5-47bc-891e-ac6d30e03694 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.294395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 353/703] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:08.300827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f2a9c-6ef5-47bc-891e-ac6d30e03694 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.303558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4747b0-02b9-4fa9-aa90-068c0022fda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.304580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4747b0-02b9-4fa9-aa90-068c0022fda1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.305835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4747b0-02b9-4fa9-aa90-068c0022fda1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.306305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4747b0-02b9-4fa9-aa90-068c0022fda1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.307201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4747b0-02b9-4fa9-aa90-068c0022fda1 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a25194d34d67429b92944e0484de03cd) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:08.307692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4747b0-02b9-4fa9-aa90-068c0022fda1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.309437 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4747b0-02b9-4fa9-aa90-068c0022fda1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.315964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 351/704] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/role_assignments?user.id=a25194d34d67429b92944e0484de03cd&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:08.317408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba0bd9-9b53-4d96-ac6f-483403b39cf6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.317915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba0bd9-9b53-4d96-ac6f-483403b39cf6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.318371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba0bd9-9b53-4d96-ac6f-483403b39cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.340327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ceba0bd9-9b53-4d96-ac6f-483403b39cf6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:08.342409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ceba0bd9-9b53-4d96-ac6f-483403b39cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:08.378830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:08.446036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 352/705] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:08.446036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba0bd9-9b53-4d96-ac6f-483403b39cf6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.449717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f2a9c-6ef5-47bc-891e-ac6d30e03694 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.504905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.505507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.505771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.506034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.518292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.518292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:08.531748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:08.532721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:08.533620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:08.536396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 354/706] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:08.538990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.580875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba0bd9-9b53-4d96-ac6f-483403b39cf6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.604306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:08.616925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.617979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.618431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.618875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.619902 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.622334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:08.634688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:08.637461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:08.639476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 353/707] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:08.645256 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.691292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7e737-db9e-4dab-b926-94f9de96ecf5 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.712214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:08.763737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.764253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.765930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.766204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.766906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:08.769039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.770498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.794039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 355/708] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:08.799980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.811992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9cdee-50e5-4d19-8d59-a183e90a79c5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.848216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.849405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.849689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.849959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.850595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:08.850892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.853300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:08.858752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 354/709] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:08.859574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:08.912714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:08.948746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f0359-4218-4846-99df-4b2c1e90fab7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.981283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08365c30-6f34-4916-84f4-dc899f1c9834 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:08.990187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:08.990782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:08.991288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:08.991288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:08.992017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=01afe498af8c42e8bca4e993897ad1d9) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:08.992347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:08.993780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:09.000316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 356/710] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/role_assignments?user.id=01afe498af8c42e8bca4e993897ad1d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:09.000459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:09.028740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d2f36-e180-4994-ab27-65ab0eb83002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:09.029391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d2f36-e180-4994-ab27-65ab0eb83002 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.029737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d2f36-e180-4994-ab27-65ab0eb83002 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.030074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d2f36-e180-4994-ab27-65ab0eb83002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.042233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d2f36-e180-4994-ab27-65ab0eb83002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:09.043961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d2f36-e180-4994-ab27-65ab0eb83002 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:09.045891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-640d2f36-e180-4994-ab27-65ab0eb83002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:09.059819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:09.061090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:09.062238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:09.062238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:09.062238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:09.062974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 355/711] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:09.064634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d4dad-128d-41c0-8d95-962ff2715e73 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.064634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d4dad-128d-41c0-8d95-962ff2715e73 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.064997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d4dad-128d-41c0-8d95-962ff2715e73 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.066624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 356/712] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:09.068462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d4dad-128d-41c0-8d95-962ff2715e73 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:09.130290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b83eb-f426-4332-b71c-eba1c42bb711 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:09.171353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f599d-b7e2-4266-9c7d-4b615b282cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:09.172215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f599d-b7e2-4266-9c7d-4b615b282cb8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.172637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f599d-b7e2-4266-9c7d-4b615b282cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.173076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f599d-b7e2-4266-9c7d-4b615b282cb8 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.174005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f599d-b7e2-4266-9c7d-4b615b282cb8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:09.174433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f599d-b7e2-4266-9c7d-4b615b282cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:09.176033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f599d-b7e2-4266-9c7d-4b615b282cb8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:09.180342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d4dad-128d-41c0-8d95-962ff2715e73 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:09.180851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:09.186252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 357/713] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Aug 12 10:38:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:09.187264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb288d-daa4-4c58-b2a0-b89d029465f3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.187544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb288d-daa4-4c58-b2a0-b89d029465f3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.187862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb288d-daa4-4c58-b2a0-b89d029465f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.210787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-59fb288d-daa4-4c58-b2a0-b89d029465f3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:09.212026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-59fb288d-daa4-4c58-b2a0-b89d029465f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:09.233849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:09.235169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.235582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.236009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.236859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:09.237318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:09.239772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:09.257978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 357/714] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:09.264015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:09.300846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 358/715] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:09.382011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a66d1-247f-4ed1-89c3-8a35d75dbb1e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:09.417557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:09.418163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.418490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.418755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.419391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:09.420846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:09.456100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 358/716] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:38:09 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:09.590314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb288d-daa4-4c58-b2a0-b89d029465f3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:09.620425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:09.709238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb288d-daa4-4c58-b2a0-b89d029465f3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:09.758542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:09.759242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.759519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.759787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] PATH_INFO: `/v3/services/network` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.760423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:09.762387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:09.773834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:09.774894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:09.777014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 359/717] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:09 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:09.781328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:09.802536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:09.891964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e3dd8-6e57-4759-86bc-803d68969706 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:09.932301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2016874-62db-4ba6-a40b-1f37b4f36bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:09.933263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2016874-62db-4ba6-a40b-1f37b4f36bbe None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:09.933713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2016874-62db-4ba6-a40b-1f37b4f36bbe None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:09.934201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2016874-62db-4ba6-a40b-1f37b4f36bbe None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:09.935016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2016874-62db-4ba6-a40b-1f37b4f36bbe None admin] RBAC: Adding query filter params (name=network) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:09.935486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2016874-62db-4ba6-a40b-1f37b4f36bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:09.937057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2016874-62db-4ba6-a40b-1f37b4f36bbe None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:09.941980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 359/718] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:09.947592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:10.050377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:10.061488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9f8ac-3ac0-4bec-92ca-feb20bb9fbd6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:10.098899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283fea6-a8fb-4bc7-adf9-50016b602cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:10.099513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283fea6-a8fb-4bc7-adf9-50016b602cab None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:10.099785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283fea6-a8fb-4bc7-adf9-50016b602cab None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:10.100051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283fea6-a8fb-4bc7-adf9-50016b602cab None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:10.100743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283fea6-a8fb-4bc7-adf9-50016b602cab None admin] RBAC: Adding query filter params (type=network) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:10.101041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283fea6-a8fb-4bc7-adf9-50016b602cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:10.102460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283fea6-a8fb-4bc7-adf9-50016b602cab None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:10.109016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 360/719] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:10.559002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1430f-42ac-4019-b2c5-7375e1dd3588 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:10.559567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1430f-42ac-4019-b2c5-7375e1dd3588 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:10.559567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1430f-42ac-4019-b2c5-7375e1dd3588 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:10.561116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 360/720] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:10.566495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b465d5-6fc9-4ce2-aab9-aaadea554470 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:10.566777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b465d5-6fc9-4ce2-aab9-aaadea554470 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:10.567045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b465d5-6fc9-4ce2-aab9-aaadea554470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:10.593306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a9b465d5-6fc9-4ce2-aab9-aaadea554470 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:10.594416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a9b465d5-6fc9-4ce2-aab9-aaadea554470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:10.674559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 361/721] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:38:11.042255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1430f-42ac-4019-b2c5-7375e1dd3588 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:11.141968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:11.171903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1430f-42ac-4019-b2c5-7375e1dd3588 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:11.222501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59945f2-5d26-48e4-9377-8911e7a55a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:11.223149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59945f2-5d26-48e4-9377-8911e7a55a20 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.223425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59945f2-5d26-48e4-9377-8911e7a55a20 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.223689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59945f2-5d26-48e4-9377-8911e7a55a20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.235271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59945f2-5d26-48e4-9377-8911e7a55a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-c59945f2-5d26-48e4-9377-8911e7a55a20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:11.240009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:11.241017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:11.241869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 361/722] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:11.247485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b465d5-6fc9-4ce2-aab9-aaadea554470 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:11.334995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:11.360040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b465d5-6fc9-4ce2-aab9-aaadea554470 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:11.405648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed174-bada-4e39-a883-82ab87572b49 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.406008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed174-bada-4e39-a883-82ab87572b49 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.406189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed174-bada-4e39-a883-82ab87572b49 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.407694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 362/723] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:11.413308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31b610-c947-413b-8315-fb8f8c0ad9bc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.413308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31b610-c947-413b-8315-fb8f8c0ad9bc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.413308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31b610-c947-413b-8315-fb8f8c0ad9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.421268 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:11.422605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.422605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.422605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.423231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:11.423538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:11.424940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:11.433288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-df31b610-c947-413b-8315-fb8f8c0ad9bc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:11.434776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-df31b610-c947-413b-8315-fb8f8c0ad9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:11.439153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 362/724] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:11.444773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:11.503322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 363/725] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:38:11.586514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7c299-4b3a-481b-9e49-ab98749c25d0 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.587108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7c299-4b3a-481b-9e49-ab98749c25d0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.587108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7c299-4b3a-481b-9e49-ab98749c25d0 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.588423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 364/726] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:11.590906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb496243-f81f-436b-b7dd-6da135fec04d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:11.595574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cadf47-ce1b-4c8b-8f14-bd3789e60f1a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.596102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cadf47-ce1b-4c8b-8f14-bd3789e60f1a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.596603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cadf47-ce1b-4c8b-8f14-bd3789e60f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.622296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f5cadf47-ce1b-4c8b-8f14-bd3789e60f1a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:11.628549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f5cadf47-ce1b-4c8b-8f14-bd3789e60f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:11.654523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:11.655868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.656475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.656953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.679621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:11.682078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:11.694545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:11.717462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:11.718497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:11.719419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:11.719419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:11.719419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:11.721272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 363/727] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:11.728311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:11.766722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 365/728] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 10:38:11.852406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cadf47-ce1b-4c8b-8f14-bd3789e60f1a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:11.855343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:11.867748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620412d-fe89-456c-bbe6-bdc5d348abb7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:11.906166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:11.907796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:11.908477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:11.908980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:11.909823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:11.910316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:11.911876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:11.925357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 364/729] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:11.930726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.024480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cadf47-ce1b-4c8b-8f14-bd3789e60f1a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.042282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:12.065657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.066655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.067155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.067617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.068546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:12.070117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:12.091106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 366/730] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:11 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:38:12.110459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb224ae5-146a-4861-a437-937a726b1796 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.131389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.185107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155070d2-2b84-407c-976b-db322b906065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.185764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155070d2-2b84-407c-976b-db322b906065 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.186040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155070d2-2b84-407c-976b-db322b906065 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.186310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155070d2-2b84-407c-976b-db322b906065 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.194704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155070d2-2b84-407c-976b-db322b906065 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:12.196190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155070d2-2b84-407c-976b-db322b906065 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-155070d2-2b84-407c-976b-db322b906065 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:12.215532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:12.216614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:12.217590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 365/731] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:38:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:12.222882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155070d2-2b84-407c-976b-db322b906065 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.241552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:12.287370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc994a-1db9-4825-a5c7-05d2793f7309 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.331406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d1e6a-6a5b-4fac-a82f-232544c4f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.332479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d1e6a-6a5b-4fac-a82f-232544c4f33a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.332973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d1e6a-6a5b-4fac-a82f-232544c4f33a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.333442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d1e6a-6a5b-4fac-a82f-232544c4f33a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.349350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d1e6a-6a5b-4fac-a82f-232544c4f33a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-ff1d1e6a-6a5b-4fac-a82f-232544c4f33a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:12.354475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:12.355407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 367/732] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:12.356919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f8aba-dfb5-41bc-a568-79bd3c8741bc None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.357681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f8aba-dfb5-41bc-a568-79bd3c8741bc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.357681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f8aba-dfb5-41bc-a568-79bd3c8741bc None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.358787 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 368/733] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:12.363258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155070d2-2b84-407c-976b-db322b906065 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.363387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f8aba-dfb5-41bc-a568-79bd3c8741bc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.379046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:12.403916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaf8f0-4032-47e2-b0bf-72e998fdcee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.404556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaf8f0-4032-47e2-b0bf-72e998fdcee0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.404938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaf8f0-4032-47e2-b0bf-72e998fdcee0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.405214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaf8f0-4032-47e2-b0bf-72e998fdcee0 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.405961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaf8f0-4032-47e2-b0bf-72e998fdcee0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:12.406266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaf8f0-4032-47e2-b0bf-72e998fdcee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:12.407705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaf8f0-4032-47e2-b0bf-72e998fdcee0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:12.426464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 366/734] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:12.427755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74e5f1-6423-4b7c-b909-9159930860d0 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.428037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74e5f1-6423-4b7c-b909-9159930860d0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.428393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74e5f1-6423-4b7c-b909-9159930860d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.447213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1f74e5f1-6423-4b7c-b909-9159930860d0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:12.448488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1f74e5f1-6423-4b7c-b909-9159930860d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:12.504276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f8aba-dfb5-41bc-a568-79bd3c8741bc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.527577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 367/735] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:12.527577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74e5f1-6423-4b7c-b909-9159930860d0 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.538451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:12.562088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.562742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.563025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.563291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.563971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:12.564283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:12.565715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:12.587071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 369/736] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:12.588033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.688336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74e5f1-6423-4b7c-b909-9159930860d0 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.689996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:12.726406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d9949-1653-4578-afca-f5ecffa666f3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.766282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d1c91-43d0-41be-8da6-39221391ed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.766916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d1c91-43d0-41be-8da6-39221391ed42 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.767189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d1c91-43d0-41be-8da6-39221391ed42 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.767451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d1c91-43d0-41be-8da6-39221391ed42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.779873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d1c91-43d0-41be-8da6-39221391ed42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:12.781443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d1c91-43d0-41be-8da6-39221391ed42 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-347d1c91-43d0-41be-8da6-39221391ed42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:12.797310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:12.798352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:12.799324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:12.799324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:12.799324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:12.799324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 370/737] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:12.804304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b1ce-b91d-432c-9a4b-5ab4501f9706 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.804304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b1ce-b91d-432c-9a4b-5ab4501f9706 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.804304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b1ce-b91d-432c-9a4b-5ab4501f9706 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.804304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 371/738] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:12.805051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084b1ce-b91d-432c-9a4b-5ab4501f9706 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.848412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:12.852932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94752b26-9f1b-454f-8f03-f58298de9cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.853754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752b26-9f1b-454f-8f03-f58298de9cb1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.853868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752b26-9f1b-454f-8f03-f58298de9cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.854088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752b26-9f1b-454f-8f03-f58298de9cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.864433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752b26-9f1b-454f-8f03-f58298de9cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:12.866090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752b26-9f1b-454f-8f03-f58298de9cb1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-94752b26-9f1b-454f-8f03-f58298de9cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:12.881557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:12.882510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:12.883404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:12.883404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:12.883404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:12.883404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 368/739] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:12.884063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14196-6328-473e-a758-98e74f9aa6be None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.884355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14196-6328-473e-a758-98e74f9aa6be None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.884624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14196-6328-473e-a758-98e74f9aa6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.902071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-47d14196-6328-473e-a758-98e74f9aa6be None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:12.903645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-47d14196-6328-473e-a758-98e74f9aa6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:12.929587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084b1ce-b91d-432c-9a4b-5ab4501f9706 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:12.966609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:12.967487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:12.967659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 369/740] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 10:38:12.968081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:12.968487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14196-6328-473e-a758-98e74f9aa6be None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:12.968705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:12.969494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:12.969965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:12.971653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:12.986901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 372/741] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:12.987869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.090407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.094337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.099344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be422f-9d56-458a-8691-5a584b387557 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.115048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14196-6328-473e-a758-98e74f9aa6be None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.145208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.146524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.147238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.147673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.148724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:13.149533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.151542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.157941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.158657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.158998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.159403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.172289 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 373/742] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:13.172289 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.179057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:13.183949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:13.184959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:13.186021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 370/743] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:13.186854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.295860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18c112-dbdd-4516-b39f-438d05f089c7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.310660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe22bc-1d4e-4e1a-8347-b684e2516aca None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.336961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.337554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.337988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.338263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.345344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.346956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.352699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.353644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.353920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.354188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:13.358302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:13.359186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 371/744] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:13.365451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.368363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.369726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.371054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:13.387261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:13.388317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 374/745] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:13.389270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.408927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.481018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5194b3a-dbea-4272-a9e0-b4a5f45af82b None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.491158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af25e50-20cb-4b63-9287-e8da2875ff1d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.519391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.520424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.520875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.521347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.522178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:13.522746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.524458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.525949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.526555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.526831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.527099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.527847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.529259 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.534861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 372/746] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:13.536011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.552490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 375/747] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:13 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:13.553299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.621581 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.670178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfceaa66-4ca0-4c2b-adcb-c67036569e78 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.672656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.711332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e40be-e106-41b3-99e7-c5ce0947f79d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.720807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.721470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.722048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.722322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.723346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:13.723655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.725342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.739034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 373/748] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:13.740123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.752532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.753501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.754022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.754481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.755407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:13.755898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.757712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.778089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 376/749] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:13.778752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.872197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e92cc5-67ad-4fd1-8185-7cac0c691135 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.883686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f8bd9-1e3a-4bc8-947b-89dc6df7b25a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:13.908676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.914252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:13.920062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.921292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.921718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.922127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.922424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:13.923231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:13.923655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:13.924065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a25194d34d67429b92944e0484de03cd/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:13.926428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=a25194d34d67429b92944e0484de03cd, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.927968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.934747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:13.936212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:13.945069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 377/750] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:38:13 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a25194d34d67429b92944e0484de03cd/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:38:13.946213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:13.971412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:13.972464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:13.973376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:13.973376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:13.973376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:13.976986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 374/751] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:13.983293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:14.131993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684dfe-57b8-475d-8ee9-d9c831e2b364 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:14.159296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e2357-d191-421d-9d5f-93b67e6baa9d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:14.177435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae819e-65ff-45e7-b0f0-ece0f7893835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:14.178044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae819e-65ff-45e7-b0f0-ece0f7893835 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.178314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae819e-65ff-45e7-b0f0-ece0f7893835 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.178577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae819e-65ff-45e7-b0f0-ece0f7893835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.199242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0454d00-09d9-4fbe-85ca-e599573d3358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:14.200030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0454d00-09d9-4fbe-85ca-e599573d3358 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.200507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0454d00-09d9-4fbe-85ca-e599573d3358 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.200869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0454d00-09d9-4fbe-85ca-e599573d3358 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.201638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0454d00-09d9-4fbe-85ca-e599573d3358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:14.202037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0454d00-09d9-4fbe-85ca-e599573d3358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:14.203521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0454d00-09d9-4fbe-85ca-e599573d3358 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:14.218504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 375/752] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:14.220684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51db363-aeff-4f6a-bd48-82a74dae1669 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.221065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51db363-aeff-4f6a-bd48-82a74dae1669 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.221433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51db363-aeff-4f6a-bd48-82a74dae1669 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.223461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 376/753] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:14.225262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51db363-aeff-4f6a-bd48-82a74dae1669 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:14.251694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:14.269282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae819e-65ff-45e7-b0f0-ece0f7893835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:14.269815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:14.271929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae819e-65ff-45e7-b0f0-ece0f7893835 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:14.349954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51db363-aeff-4f6a-bd48-82a74dae1669 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:14.384197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 378/754] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:14.385532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e135-aeea-44a6-a8e5-92fa883afced None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.385809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e135-aeea-44a6-a8e5-92fa883afced None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.386084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8e135-aeea-44a6-a8e5-92fa883afced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.388617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:14.389591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.390083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.390544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.398621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:14.400489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:14.404836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b9c8e135-aeea-44a6-a8e5-92fa883afced None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:14.406040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b9c8e135-aeea-44a6-a8e5-92fa883afced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:14.414821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:14.415817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:14.416810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:14.419613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 377/755] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:38:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:14.420811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:14.480153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 379/756] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:14.481234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8e135-aeea-44a6-a8e5-92fa883afced None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:14.504248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:14.528116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:14.533451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652304ce-a978-46d9-a1dc-83640c8206a9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:14.570821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:14.571835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.572346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.572832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.584796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:14.586727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:14.602219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:14.603212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:14.604207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:14.604207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:14.604207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:14.604419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8e135-aeea-44a6-a8e5-92fa883afced None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:14.606640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 378/757] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:14.617386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:14.642147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:14.642673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.642947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.643272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.644255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:14.644582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:14.646635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:14.666309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 380/758] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Aug 12 10:38:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:14.672510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:14.815449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158c3ee-bcbb-48fb-b308-97cf381a5656 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:14.825261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:14.857268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b126-e3cc-4954-bbdc-0de9b2d9c5e8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:14.865225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:14.866070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.866568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.867033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.867953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:14.868488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:14.870082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:14.882516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 379/759] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:14.883740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:14.892779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:14.893313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:14.893587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:14.893850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:14.897113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:14.966340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:14.968091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:14.985123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53abdf-7a30-4764-990a-ec11a38a0457 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.021114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.021936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.022307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.022672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.023397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.024918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:15.036565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:15.037598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:15.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 380/760] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Aug 12 10:38:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:15.039216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.059441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.084773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 381/761] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:15.085889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.130365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.162573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051fc46-da26-45d1-ba8d-dd222000b983 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.206290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.206937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.207341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.207731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.212285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a1d1f-a423-4cb2-a77c-c4b585486746 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.216576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.218057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:15.233120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:15.234289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:15.235391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 381/762] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:15.236008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.253766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.254549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.254936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.255308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.256051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:15.256474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.257979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.268022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 382/763] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:15.269052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.332656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.361725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05179c8-932f-48ad-8c1c-49a973478dd9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.390550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8e26c-ee37-4a2e-b9c9-096f43c77c65 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.404002 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.404662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.405303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.405578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.407918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.419942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.421413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.432900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.433825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.434322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.434794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.435700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:15.436200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:15.436417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:15.437337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:15.438259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:15.438259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:15.438259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:15.440307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.442301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 382/764] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:15.443459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.460476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 383/765] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:15.461911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.549464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7905c18-acb7-4c47-ae69-8232317e569c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.586217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.586831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.587154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.587427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.588156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:15.588520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.590020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.595633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d27e47-58bd-4a79-8bfd-d67ca08058be None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.597247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.599579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 383/766] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:15.600666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.637815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975c89b-b9ca-4293-9812-e446adcd5de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.638423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975c89b-b9ca-4293-9812-e446adcd5de1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.638699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975c89b-b9ca-4293-9812-e446adcd5de1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.638969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975c89b-b9ca-4293-9812-e446adcd5de1 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.639609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975c89b-b9ca-4293-9812-e446adcd5de1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:15.639911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975c89b-b9ca-4293-9812-e446adcd5de1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.641325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975c89b-b9ca-4293-9812-e446adcd5de1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.652990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 384/767] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:15.658512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587079d-00b6-4afd-9ce0-c69ce40c7629 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.658801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587079d-00b6-4afd-9ce0-c69ce40c7629 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.659065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587079d-00b6-4afd-9ce0-c69ce40c7629 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.660348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 385/768] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:15.661053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587079d-00b6-4afd-9ce0-c69ce40c7629 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.666145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.718826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dc438-2914-4775-ab05-23f96a482f35 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.754192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.755354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.755815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.756082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.756833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=01afe498af8c42e8bca4e993897ad1d9) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:15.757136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.758568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.765719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 384/769] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/role_assignments?user.id=01afe498af8c42e8bca4e993897ad1d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:15.769208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.797267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587079d-00b6-4afd-9ce0-c69ce40c7629 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.849964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df367a-4272-4d9e-b5ff-2a5533114f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.851764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df367a-4272-4d9e-b5ff-2a5533114f93 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.852300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df367a-4272-4d9e-b5ff-2a5533114f93 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.853029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df367a-4272-4d9e-b5ff-2a5533114f93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.855024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df367a-4272-4d9e-b5ff-2a5533114f93 None admin] RBAC: Adding query filter params (interface=public, service_id=f6a96164b9134476af859bb2040336e9, region_id=RegionOne) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:15.855599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df367a-4272-4d9e-b5ff-2a5533114f93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.857612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df367a-4272-4d9e-b5ff-2a5533114f93 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.869552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 386/770] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/endpoints?service_id=f6a96164b9134476af859bb2040336e9&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:15.872274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42378e-cdf6-4363-b24d-ae33562aede1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.872704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42378e-cdf6-4363-b24d-ae33562aede1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.873115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42378e-cdf6-4363-b24d-ae33562aede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.892281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a19a-aa51-42f5-ac32-fe665472e858 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:15.896429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cd42378e-cdf6-4363-b24d-ae33562aede1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:15.897053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cd42378e-cdf6-4363-b24d-ae33562aede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:15.912234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.923209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:15.936732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:15.937511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:15.937902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:15.938262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:15.945423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:15.946990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:15.958712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:15.959811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:15.960960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:15.967148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 385/771] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:15.968138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:15.974258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 387/772] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:15.975040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42378e-cdf6-4363-b24d-ae33562aede1 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.086350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f4d7f-234e-49d9-aa1a-dd19e044d1e0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.129712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42378e-cdf6-4363-b24d-ae33562aede1 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.130696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43002b6-eba7-4b1d-84f7-f4fc7959375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.131611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43002b6-eba7-4b1d-84f7-f4fc7959375a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.132452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43002b6-eba7-4b1d-84f7-f4fc7959375a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.132748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43002b6-eba7-4b1d-84f7-f4fc7959375a None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.133364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43002b6-eba7-4b1d-84f7-f4fc7959375a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:16.134737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43002b6-eba7-4b1d-84f7-f4fc7959375a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:16.144945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 386/773] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:16.146377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5bdaf-fe0e-4146-abcd-3988b126fb5e None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.146795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5bdaf-fe0e-4146-abcd-3988b126fb5e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.147189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5bdaf-fe0e-4146-abcd-3988b126fb5e None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.148810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 387/774] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:16.161432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d315-e753-4c0a-98bb-9f9d967f1931 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.161941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d315-e753-4c0a-98bb-9f9d967f1931 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.162355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d315-e753-4c0a-98bb-9f9d967f1931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.178733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.179639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.179922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.180188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.181919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:16.182232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:16.183764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:16.198121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 388/775] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:16.204230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.205491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3634d315-e753-4c0a-98bb-9f9d967f1931 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:16.206836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3634d315-e753-4c0a-98bb-9f9d967f1931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:16.242219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:16.250137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:16.306709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 388/776] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:16.307623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634d315-e753-4c0a-98bb-9f9d967f1931 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.369382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af1583-6681-427f-bbec-97313c6f9c5a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.410126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.410727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.411002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.411270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.411993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=e8008fa945f540a18aac7b0f81edf0d9) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:16.412321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:16.413813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:16.419897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 389/777] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:16 2022] GET /identity/v3/role_assignments?user.id=e8008fa945f540a18aac7b0f81edf0d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:16.434051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634d315-e753-4c0a-98bb-9f9d967f1931 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.473096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.473721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.473996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.474260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] PATH_INFO: `/v3/services/network` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.474847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:16.476208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:16.485521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:16.486539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:16.487368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 389/778] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:16 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:16.492907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.582719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:16.604235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad8776-5294-4207-a6da-b8fb71263d9e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.635539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.646581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.647185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.647460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.647727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.648381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] RBAC: Adding query filter params (name=network) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:16.648678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:16.650048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:16.657217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 390/779] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:16.665313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.681248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:16.770205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e432-073e-4971-9c20-5b7709eb1d08 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.773408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08984cae-8c98-4c0f-b34e-fbbbaf9c431a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.817005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.817630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.817901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.818171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.818803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] RBAC: Adding query filter params (type=network) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:16.819102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:16.820514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:16.827762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 391/780] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:16.832924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.833538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.833672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.833859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.834091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.855330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:16.855964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:16.862351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:16.863902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:16.865223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 390/781] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:16.873451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:16.952833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fd3a4-c820-4111-93b1-b18a0995f52d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:16.997795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:16.998409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:16.998686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:16.998968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:16.999657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] RBAC: Adding query filter params (interface=public, service_id=d345ebac08fe414ea0d36f4371cbf942, region_id=RegionOne) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:16.999959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:17.001361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:17.016678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 392/782] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:38:16 2022] GET /identity/v3/endpoints?service_id=d345ebac08fe414ea0d36f4371cbf942&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:17.021906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:17.047136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:17.066977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9093091-6bf9-45fb-9d9d-d7d2b87d140c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:17.136982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:17.162714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4c56f-a030-489c-8a6e-3588044f5c49 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:17.163710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:17.164593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:17.165064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:17.165540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:17.166403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:17.166903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:17.168682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:17.198509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 391/783] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:17.198708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:17.199205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:17.199647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:17.199953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:17.200589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:17.201958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:17.203975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:17.207445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 393/784] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:17 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:17.357872 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3827be-b07d-4a4e-9bb0-756b2d813a46 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:17.397081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:17.397734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:17.398005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:17.398270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:17.411011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:17.412529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:17.428743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:17.429780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:17.431269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:17.431269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:17.431269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:17.431269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 392/785] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:17.437434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:17.549716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fe763-92af-4266-b4b2-99411817f0d0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:17.592059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:17.592619 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:17.592893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:17.593159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:17.593857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:17.594160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:17.595528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:17.597928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:17.612183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 394/786] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:17.618100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:17.629337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:17.742026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd036a90-1383-40f2-81b9-a781e552af44 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:17.793080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:17.794238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:17.794736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:17.795192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:17.808057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:17.809912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:17.826023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:17.827040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:17.827979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:17.831465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 393/787] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:38:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:17.836524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:18.000048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b29d-ac38-4f4d-b80c-4c47ac1b36f3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:18.045978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c88a9-6751-46dd-a92f-a191a600b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:18.046594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c88a9-6751-46dd-a92f-a191a600b7bb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.046865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c88a9-6751-46dd-a92f-a191a600b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.047130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c88a9-6751-46dd-a92f-a191a600b7bb None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.047858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c88a9-6751-46dd-a92f-a191a600b7bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:18.048156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c88a9-6751-46dd-a92f-a191a600b7bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:18.049614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c88a9-6751-46dd-a92f-a191a600b7bb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:18.071502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 395/788] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:38:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:18.076976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:18.126170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1abe7f-a514-4a04-bd26-67b12d67c0e4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.126493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1abe7f-a514-4a04-bd26-67b12d67c0e4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.126723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1abe7f-a514-4a04-bd26-67b12d67c0e4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.128772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 396/789] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:18.133502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e394d-d68f-4320-81aa-571368213bbd None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.133781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e394d-d68f-4320-81aa-571368213bbd None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.134051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e394d-d68f-4320-81aa-571368213bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.155087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a30e394d-d68f-4320-81aa-571368213bbd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:18.156327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a30e394d-d68f-4320-81aa-571368213bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:18.196711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:18.264412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:18.266639 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdd39c-cd40-4114-87c4-7fe1fcf64c1d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:18.302485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 397/790] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:18.314593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:18.315534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.316004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.316524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.399188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:18.401047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:18.429056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b65c-0e20-43ca-ad1e-e2adc837439b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.429679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b65c-0e20-43ca-ad1e-e2adc837439b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.430154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6b65c-0e20-43ca-ad1e-e2adc837439b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.431917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 398/791] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:18.437041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80d28e-f6b8-474d-aaaa-2f14208414e2 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.437559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80d28e-f6b8-474d-aaaa-2f14208414e2 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.438015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80d28e-f6b8-474d-aaaa-2f14208414e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.456388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-1b80d28e-f6b8-474d-aaaa-2f14208414e2 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:18.457897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1b80d28e-f6b8-474d-aaaa-2f14208414e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:18.471370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:18.540590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 394/792] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:18.547104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:18.576495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 399/793] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:18.616674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80d28e-f6b8-474d-aaaa-2f14208414e2 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:18.726417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f893672-07f9-4cd6-8515-be99e53bdcb7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:18.776241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80d28e-f6b8-474d-aaaa-2f14208414e2 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:18.793102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:18.797454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:18.798459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.798947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.799400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.818139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:18.819880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:18.820361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8a2c7-5965-4735-b3f4-80731a49feaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:18.821200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8a2c7-5965-4735-b3f4-80731a49feaf None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.821479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8a2c7-5965-4735-b3f4-80731a49feaf None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.821752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8a2c7-5965-4735-b3f4-80731a49feaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.834784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8a2c7-5965-4735-b3f4-80731a49feaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:18.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:18.837388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:18.838312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:18.838312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:18.838312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2bb8a2c7-5965-4735-b3f4-80731a49feaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:18.839493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:18.840598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:18.841994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 400/794] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:18.842701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b57ad1-9137-482f-9a06-1a5518e1c06d None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.842701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b57ad1-9137-482f-9a06-1a5518e1c06d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.842824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b57ad1-9137-482f-9a06-1a5518e1c06d None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.847837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 395/796] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:18.848348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 401/796] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:18.849386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:18.849712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeff1a-76de-459b-b9ac-d6511d639832 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:18.850054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeff1a-76de-459b-b9ac-d6511d639832 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:18.850883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeff1a-76de-459b-b9ac-d6511d639832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:18.870043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d0aeff1a-76de-459b-b9ac-d6511d639832 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:18.871483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d0aeff1a-76de-459b-b9ac-d6511d639832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:18.877040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:18.942659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 402/797] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:18.943291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeff1a-76de-459b-b9ac-d6511d639832 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:18.973544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdea5a-6438-4d31-9687-76b5852f2522 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.021407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.022331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.022781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.023289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.024222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.024714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.026492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.036185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 396/798] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.037449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.082935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeff1a-76de-459b-b9ac-d6511d639832 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.085788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.127743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.130542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.131157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.131657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.131938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.132635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.132925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.134589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.154431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 403/799] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.154431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.156602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5f64-8704-4141-97d7-eb3a35a31f3a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.193765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.194749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.195215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.195672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.196522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.198204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:19.207984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:19.209022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 10:38:19.210434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:19.212505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 397/800] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:19.213555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.266061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d717fa-baf7-4075-81cc-1c1a5e609e9b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.301672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.302294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.302562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.302822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.314270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.315703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.316417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f838a-cca1-44c4-ac97-5cf12139e4ef None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:19.330168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:19.331183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:19.332199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:19.332199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:19.332199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:19.332199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 404/801] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:19.332199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.354106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.361108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.361765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.362035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.362298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.369185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.370630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:19.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:19.383337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:19.384225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:19.388627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 398/802] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:19.389498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.393304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.451170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71d6f-1da1-45b8-80bb-b9fd05b0e888 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.484578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.485199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.485477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.485747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.493830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbcb16-e07f-44dc-9d5e-cbff19f4e523 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.496854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:19.501939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:19.502897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:19.503769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 405/803] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:19.504515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.533048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.534056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.534505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.534991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.535816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.536319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.538077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.547709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 399/804] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.548879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.584069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.608560 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab1b4-65ae-4db6-bd30-29a90b2687a0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.618622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.644269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.644904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.645176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.645454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.646078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.646371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.647761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.655207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8ca65-78c0-40d0-9bc1-d4bd1a10defb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.661620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 406/805] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.662674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.692041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.692670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.692945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.693211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.693898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.694197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.695632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.712363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 400/806] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.712363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716434e-4787-4472-843b-8b538e892df8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.803547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.804144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.804442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.804694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.805343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.805638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.806994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.807371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da90c-815f-48fe-add8-936c86b8e5b7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.809281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.816618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 407/807] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.817416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.835342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:19.843364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.843980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.844245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.844539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.845159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.845455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.846824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.851550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 401/808] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.852425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.920543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a1524-1364-4697-908c-1ab2819e42f8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.948034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196491e-bf5a-4c6e-bdb7-5b0916968d88 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:19.952975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adf4b8-d5a8-46ce-8ad0-c9bd5506b9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.953694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf4b8-d5a8-46ce-8ad0-c9bd5506b9ae None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.953959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf4b8-d5a8-46ce-8ad0-c9bd5506b9ae None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.954220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf4b8-d5a8-46ce-8ad0-c9bd5506b9ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.961095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf4b8-d5a8-46ce-8ad0-c9bd5506b9ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.962528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf4b8-d5a8-46ce-8ad0-c9bd5506b9ae None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-71adf4b8-d5a8-46ce-8ad0-c9bd5506b9ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:19.973751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:19.974647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:19.975580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:19.976634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 408/809] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:19.977675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d44cb1-02ac-4954-aa7a-27da0f94248e None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.977950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d44cb1-02ac-4954-aa7a-27da0f94248e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.978222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d44cb1-02ac-4954-aa7a-27da0f94248e None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.979430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 409/810] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:19.980100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d44cb1-02ac-4954-aa7a-27da0f94248e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:19.984699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:19.985301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:19.985571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:19.985836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:19.986557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a25194d34d67429b92944e0484de03cd) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:19.986860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:19.988296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:19.994537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 402/811] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/role_assignments?user.id=a25194d34d67429b92944e0484de03cd&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:19.995249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.051412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.055846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.104642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656eee8-8046-49c7-8513-e7fdedbea7c3 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.107224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d44cb1-02ac-4954-aa7a-27da0f94248e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.149191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.149834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce6d8e-f296-4abf-b5b2-a9f74d8bf69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.149948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.150124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.150330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.150425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce6d8e-f296-4abf-b5b2-a9f74d8bf69e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.150650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce6d8e-f296-4abf-b5b2-a9f74d8bf69e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.150914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce6d8e-f296-4abf-b5b2-a9f74d8bf69e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.151004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.152509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.161630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce6d8e-f296-4abf-b5b2-a9f74d8bf69e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.163069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce6d8e-f296-4abf-b5b2-a9f74d8bf69e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.177458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 403/812] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:20.178201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-d3ce6d8e-f296-4abf-b5b2-a9f74d8bf69e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:20.180302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:20.181251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:20.182214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:20.182214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:20.182214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:20.182214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 410/813] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:20.183074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df2a2f-16b4-47f0-83f5-56460e8398a4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.183261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df2a2f-16b4-47f0-83f5-56460e8398a4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.184420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df2a2f-16b4-47f0-83f5-56460e8398a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.210169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f8df2a2f-16b4-47f0-83f5-56460e8398a4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:20.211782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f8df2a2f-16b4-47f0-83f5-56460e8398a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:20.307448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 411/814] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:20.308946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df2a2f-16b4-47f0-83f5-56460e8398a4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.316713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d6367-6a48-42ed-a9cf-93f95383474f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.334931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.351268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.364548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.365159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.365432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.365698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.366432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:20.366829 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.368400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.392413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 404/815] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:20.394104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.422530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df2a2f-16b4-47f0-83f5-56460e8398a4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.460995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.461518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.461791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.462057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.462697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:20.463087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.464722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.477968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 412/816] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:20.483141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.499039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9801ec4-95b1-4c10-9a3a-97970be84584 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.537593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.538367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.538731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.539046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.550088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.551808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:20.567271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:20.569474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:20.571560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:20.571560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:20.571560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:20.571560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 405/817] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:20.575177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.586636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.593270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d577230-be6b-4a49-9d4c-61ee13754f98 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.605614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.635672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.638440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.639186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.639669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.650424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.653495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:20.674631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:20.675672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:20.676661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:20.678109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 413/818] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:20.679695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.697262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd6714-8dce-44f5-b2a3-f68b0ffd0b8f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.735507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.736587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.737089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.737553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.738391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:20.738986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.740702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.761232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 406/819] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:20.762872 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.797010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1bc91-4e26-4525-8267-953215fd5434 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.833045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.834015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.834512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.834999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] PATH_INFO: `/v3/services/network` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.835867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.837645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:20.847241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:20.848299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 10:38:20.849240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:20.856603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 414/820] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:20.857130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.858463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.876519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55909510-3c89-4c9a-a664-ce815b286619 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:20.885324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:20.919323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:20.919925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:20.920196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:20.920904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:20.921960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:20.922266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:20.924204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:20.946772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 407/821] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:20.947998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:20.968146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391e3f9-39c3-4add-a3b7-6b0c58481980 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.002642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.003290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.003624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.003956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.011041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.013207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:21.025027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:21.025942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:21.026835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:21.031395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 415/822] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:21.032345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.059254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eea54-d4f5-47b4-a5ef-05457ed84211 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.093878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.094486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.094756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.095074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.095707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] RBAC: Adding query filter params (name=network) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:21.096004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.097538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.107237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 408/823] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:21.108207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.113497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:21.134394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11426b3e-3d8f-4510-a90d-53082110aaf8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.152640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:21.171414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.172132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.172436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.172700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.184604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.186119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:21.201338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:21.202317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:21.203222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:21.203222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:21.203222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:21.203222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 416/824] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:21.203858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.210303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba34bcc-a23a-4f2a-a95f-a7a912667a50 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.246762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.247369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.247640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.247958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.248701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:21.249048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.250488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.263431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 409/825] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:21.264298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.317012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409f637-44c2-4cd9-85ba-b4d0128349dc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.325786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:21.351299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:21.358598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.359475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.359884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.360391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.361244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] RBAC: Adding query filter params (type=network) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:21.361672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.363257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.369171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 417/826] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:21.370109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.375098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5d5f-eb22-4a5e-a3d1-41db2280b7cb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.410168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.410774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.411047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.411314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.411953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:21.412252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.413648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.426301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 410/827] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:21.427213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.479383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b74fd-95d6-41bc-996d-b76bd48c0cfc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.516669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.517639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.518048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.518473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/01afe498af8c42e8bca4e993897ad1d9/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.520628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=01afe498af8c42e8bca4e993897ad1d9, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.522352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.530446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469340ab-20c9-460b-9256-54e84c516583 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.544448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 418/828] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:38:21 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/01afe498af8c42e8bca4e993897ad1d9/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:38:21.545919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.565014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:21.568935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.569678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.569952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.570226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.570953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.572383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.601374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 411/829] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:21 2022] POST /identity/v3/endpoints => generated 337 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:21.660969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:21.724033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6237a-a37c-4fcf-af96-16efb21431ad None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.778258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.779223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.779670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.780112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.788471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.790246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:21.802980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:21.803955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:21.804976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:21.807207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 419/830] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:21.813423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:21.921718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721dcdd7-8876-45b1-9e8a-c92caf867f51 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:21.957276 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5a384-325e-4854-a0a4-4a30a652236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:21.958200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5a384-325e-4854-a0a4-4a30a652236c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:21.958611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5a384-325e-4854-a0a4-4a30a652236c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:21.959035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5a384-325e-4854-a0a4-4a30a652236c None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:21.959767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5a384-325e-4854-a0a4-4a30a652236c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:21.960068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5a384-325e-4854-a0a4-4a30a652236c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:21.961612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5a384-325e-4854-a0a4-4a30a652236c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:21.975348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 412/831] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:21.981127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:22.086520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd0b2c-ca24-44a1-aa7a-7e229112a711 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:22.100420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:22.122127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d421b-f6ea-4842-b5f0-98d19d1da7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:22.122883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d421b-f6ea-4842-b5f0-98d19d1da7d1 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.123221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d421b-f6ea-4842-b5f0-98d19d1da7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.123552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d421b-f6ea-4842-b5f0-98d19d1da7d1 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/e8008fa945f540a18aac7b0f81edf0d9/roles/db7026479a164eec9e413091ad5440fe` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.139202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d421b-f6ea-4842-b5f0-98d19d1da7d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=e8008fa945f540a18aac7b0f81edf0d9, role_id=db7026479a164eec9e413091ad5440fe)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:22.140769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d421b-f6ea-4842-b5f0-98d19d1da7d1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:22.153136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb72b-3d4f-4aea-93a9-c1ba134cf2ee None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.153638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb72b-3d4f-4aea-93a9-c1ba134cf2ee None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.154212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb72b-3d4f-4aea-93a9-c1ba134cf2ee None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.156191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 413/832] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:22.158479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 420/833] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:38:21 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/e8008fa945f540a18aac7b0f81edf0d9/roles/db7026479a164eec9e413091ad5440fe => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:38:22.161199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348bcca-985a-4401-91b2-4419a25ecd0a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.161574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348bcca-985a-4401-91b2-4419a25ecd0a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.161958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348bcca-985a-4401-91b2-4419a25ecd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.181936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7348bcca-985a-4401-91b2-4419a25ecd0a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:22.183288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7348bcca-985a-4401-91b2-4419a25ecd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:22.223173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:22.262297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 414/834] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:22.444649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6479-9521-4665-a011-dbbb3e8b2636 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.444927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6479-9521-4665-a011-dbbb3e8b2636 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.445201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6479-9521-4665-a011-dbbb3e8b2636 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.446769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 421/835] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:22.452212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86489-6b0b-4e09-ab3b-d8844b936830 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.452529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86489-6b0b-4e09-ab3b-d8844b936830 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.452803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86489-6b0b-4e09-ab3b-d8844b936830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.471077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2af86489-6b0b-4e09-ab3b-d8844b936830 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:22.472289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2af86489-6b0b-4e09-ab3b-d8844b936830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:22.530559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6479-9521-4665-a011-dbbb3e8b2636 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:22.544627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 415/836] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:22.642136 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6479-9521-4665-a011-dbbb3e8b2636 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:22.676552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:22.677176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.677447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.677710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.678286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:22.679637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:22.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:22.689426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:22.690685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 422/837] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Aug 12 10:38:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:22.695692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af86489-6b0b-4e09-ab3b-d8844b936830 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:22.792619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:22.796779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af86489-6b0b-4e09-ab3b-d8844b936830 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:22.809344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:22.837396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:22.838368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.838820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.839259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.840167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:22.840704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:22.842121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:22.842475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:22.848939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 416/838] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:22.854204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:22.931644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519bec8-001b-4ae7-a9d4-a74ac4a43d89 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:22.971515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:22.972623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:22.973098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:22.973562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:22.974362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:22.975957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:22.982432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600908a-3913-4c56-8432-72b170adadd8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:22.985641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:22.986644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:22.987522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:22.994991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 423/839] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:22.996087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:23.027823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:23.028743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:23.029024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:23.029289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:23.029911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:23.030200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:23.032190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:23.048361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 417/840] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:23.101545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:23.112761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a65b9-1b55-4e4d-8602-049c2879a733 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:23.149935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dce9f0-7bd7-40d2-b16e-fc1d8bb80a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:23.151351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce9f0-7bd7-40d2-b16e-fc1d8bb80a37 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:23.151907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce9f0-7bd7-40d2-b16e-fc1d8bb80a37 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:23.152512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce9f0-7bd7-40d2-b16e-fc1d8bb80a37 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:23.153362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce9f0-7bd7-40d2-b16e-fc1d8bb80a37 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:23.153877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce9f0-7bd7-40d2-b16e-fc1d8bb80a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:23.155477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce9f0-7bd7-40d2-b16e-fc1d8bb80a37 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:23.161442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 424/841] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:23.167507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:23.200063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:23.274553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04650661-2f0d-4071-8f06-7ca82ff98c70 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:23.309536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be747313-a987-4267-a8cb-175b3c252b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:23.310152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be747313-a987-4267-a8cb-175b3c252b85 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:23.310481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be747313-a987-4267-a8cb-175b3c252b85 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:23.310752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be747313-a987-4267-a8cb-175b3c252b85 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:23.311427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be747313-a987-4267-a8cb-175b3c252b85 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:23.311791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be747313-a987-4267-a8cb-175b3c252b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:23.313229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be747313-a987-4267-a8cb-175b3c252b85 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:23.318863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 418/842] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:23.814086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517938c-c8af-49be-bed6-c80e0538bb91 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:23.814478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517938c-c8af-49be-bed6-c80e0538bb91 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:23.814670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517938c-c8af-49be-bed6-c80e0538bb91 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:23.816218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 425/843] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:23.821367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf378-7711-47b7-9939-44d9c862dd1e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:23.821644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf378-7711-47b7-9939-44d9c862dd1e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:23.821914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf378-7711-47b7-9939-44d9c862dd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:23.841183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-deccf378-7711-47b7-9939-44d9c862dd1e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:23.842362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-deccf378-7711-47b7-9939-44d9c862dd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:23.910425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 419/844] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:24.167135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9f123-41ff-4d9e-af5b-6aef2b5eadff None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.167726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9f123-41ff-4d9e-af5b-6aef2b5eadff None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.167726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9f123-41ff-4d9e-af5b-6aef2b5eadff None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.169842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 426/845] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:24.175356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a194a0-b9da-441f-924c-2e9961493e3c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.176167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a194a0-b9da-441f-924c-2e9961493e3c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.176665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a194a0-b9da-441f-924c-2e9961493e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.192414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9f123-41ff-4d9e-af5b-6aef2b5eadff None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:24.196904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-68a194a0-b9da-441f-924c-2e9961493e3c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:24.198132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-68a194a0-b9da-441f-924c-2e9961493e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:24.216726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:24.278007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 420/846] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:24.322417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9f123-41ff-4d9e-af5b-6aef2b5eadff None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:24.357418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:24.358041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.358313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.358579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.369107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:24.373815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:24.374743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:24.375583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 427/847] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:24.382690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a194a0-b9da-441f-924c-2e9961493e3c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:24.493618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a194a0-b9da-441f-924c-2e9961493e3c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:24.531316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:24.532298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.532770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.533233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.534111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:24.534635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:24.536222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:24.546451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 421/848] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:24.552372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:24.555057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:24.575648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:24.584646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:24.673696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ef4a1-c255-44e3-9c10-03b66ea49601 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:24.674619 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef7f0-17a0-4c9a-a47c-b41a5bbdbbb6 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:24.709956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:24.710985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.711491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.711963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.716063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:24.716749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.717027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.717297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.724898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:24.728496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:24.729963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:24.731227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:24.732295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:38:24.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:24.735675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 422/849] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:24.742646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:24.751777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:24.753357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:24.754234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:24.754234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:24.754234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:24.754234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 428/850] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:24.760317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:24.829854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:24.831883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:24.861335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9c959-2f8e-44ae-a514-02bd89adb7d0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:24.867663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806885ef-e80a-4600-8d3d-8ba354c1ffd4 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:24.897780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:24.898750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.899213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.899674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.900629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:24.901114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:24.902818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:24.904792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:24.905399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:24.905677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:24.905947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:24.906585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:24.906887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:24.908277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:24.913408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 423/851] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:24.920881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:24.930123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 429/852] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:24.931285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.031531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b919-ef65-45b8-ad3b-024170248e84 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.033853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cd896-4230-4475-ad9a-3b1b36718639 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.059351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.066868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dccee-4ad4-4e4b-a477-48d0fecc1f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.067491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dccee-4ad4-4e4b-a477-48d0fecc1f13 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.067759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dccee-4ad4-4e4b-a477-48d0fecc1f13 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.068025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dccee-4ad4-4e4b-a477-48d0fecc1f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.069363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d9be2-8499-4bf6-9ee0-4ed3ac326cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.069984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d9be2-8499-4bf6-9ee0-4ed3ac326cb7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.070255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d9be2-8499-4bf6-9ee0-4ed3ac326cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.070519 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d9be2-8499-4bf6-9ee0-4ed3ac326cb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.071284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.079351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d9be2-8499-4bf6-9ee0-4ed3ac326cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.080840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d9be2-8499-4bf6-9ee0-4ed3ac326cb7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.080955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dccee-4ad4-4e4b-a477-48d0fecc1f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.082557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dccee-4ad4-4e4b-a477-48d0fecc1f13 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-0e0d9be2-8499-4bf6-9ee0-4ed3ac326cb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:25.092252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:25.093347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 430/853] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746ffd9-ccd0-4fbb-81d9-6cdf210418d1 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.099249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746ffd9-ccd0-4fbb-81d9-6cdf210418d1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.099964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746ffd9-ccd0-4fbb-81d9-6cdf210418d1 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.100816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 431/854] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-635dccee-4ad4-4e4b-a477-48d0fecc1f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:25.101540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:25.102666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:25.103577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:25.103577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:25.103577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:25.103747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746ffd9-ccd0-4fbb-81d9-6cdf210418d1 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.103826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 424/855] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:25.108759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fde8b-a5a8-4e36-ab1e-198396c67b94 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.108759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fde8b-a5a8-4e36-ab1e-198396c67b94 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.108759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fde8b-a5a8-4e36-ab1e-198396c67b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.125500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-cd0fde8b-a5a8-4e36-ab1e-198396c67b94 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:25.126696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cd0fde8b-a5a8-4e36-ab1e-198396c67b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:25.206253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 425/856] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:25.207771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fde8b-a5a8-4e36-ab1e-198396c67b94 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.222390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746ffd9-ccd0-4fbb-81d9-6cdf210418d1 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.272688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b07a72-5c3b-496c-af26-1f5fe452f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.273294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07a72-5c3b-496c-af26-1f5fe452f457 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.273569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07a72-5c3b-496c-af26-1f5fe452f457 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.273839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07a72-5c3b-496c-af26-1f5fe452f457 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.274572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b07a72-5c3b-496c-af26-1f5fe452f457 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:25.274874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b07a72-5c3b-496c-af26-1f5fe452f457 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.276334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b07a72-5c3b-496c-af26-1f5fe452f457 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.294272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 432/857] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Aug 12 10:38:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:25.295419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f55fb6-bf2a-451d-8958-9d34519ca8e8 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.295701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f55fb6-bf2a-451d-8958-9d34519ca8e8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.295967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f55fb6-bf2a-451d-8958-9d34519ca8e8 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.297303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 433/858] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:25.299695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f55fb6-bf2a-451d-8958-9d34519ca8e8 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.322408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fde8b-a5a8-4e36-ab1e-198396c67b94 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.349711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.350235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.374409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3874f46-317c-4452-8bbf-4f16972d069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.375013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874f46-317c-4452-8bbf-4f16972d069b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.375320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874f46-317c-4452-8bbf-4f16972d069b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.375584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874f46-317c-4452-8bbf-4f16972d069b None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.376167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874f46-317c-4452-8bbf-4f16972d069b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:25.376490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874f46-317c-4452-8bbf-4f16972d069b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.378256 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874f46-317c-4452-8bbf-4f16972d069b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.395616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 426/859] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:25.395616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27592-85f4-4d43-baca-0c8d52fd5b21 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.395616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27592-85f4-4d43-baca-0c8d52fd5b21 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.395616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27592-85f4-4d43-baca-0c8d52fd5b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.412073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-dba27592-85f4-4d43-baca-0c8d52fd5b21 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:25.413279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-dba27592-85f4-4d43-baca-0c8d52fd5b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:25.419552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f55fb6-bf2a-451d-8958-9d34519ca8e8 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.472515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.473148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.473423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.473694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.487947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 427/860] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:25.488811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba27592-85f4-4d43-baca-0c8d52fd5b21 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.546673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.548021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.593456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.599501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba27592-85f4-4d43-baca-0c8d52fd5b21 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.612787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.639817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.640517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.640783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.640972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.647873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.649393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.653219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 434/861] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 12 10:38:25.658400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:25.660827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:25.661776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 428/862] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:38:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:25.662759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.764874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3281f-b841-4e0c-823d-77d82982644a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.765633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a61774-fd7a-4a0e-8257-f5438af71674 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.800518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.800704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.800831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.800831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.800831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.801031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.801031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.801031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.801031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.803135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.812169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.813778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:25.820714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 435/863] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:25 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:25.821532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.825242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:25.830097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:25.831149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:25.832042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:25.832042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:25.832042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:25.832777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 429/864] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:25.833335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:25.859634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:25.948840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e4329-80fc-4eee-9d70-2d04ae7100e4 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.959274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d450-e651-4ab4-b3e5-23c80ba29d28 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:25.988965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:25.989173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:25.989422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:25.990118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:25.990317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:25.992026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.000302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.000489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.000812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.000990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.001794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:26.002082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.003651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.013085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 430/865] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:25 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:26.013930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.031225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 436/866] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Aug 12 10:38:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:26.037388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.103506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:26.106840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:26.137164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2146b-0cee-4e53-b871-8b62aae4449e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.155693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776c0c-ab8f-4c92-be14-e1928b088639 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.191369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.192130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.192724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.193000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.193795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:26.194089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.195494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.198134 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.198710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.199620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.199620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.209610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 431/867] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:26.216344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.280557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.282764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.349285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea9b21-0e21-4cd9-8ddc-69e787372ad5 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.352621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:26.393181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 437/868] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:26.396409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.397220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.397618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.398009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.398712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.404831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:26.410917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.413264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:26.429797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:26.430715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:26.431632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:26.434277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 432/869] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:26.440645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.509222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653ef6-178f-432e-bd4c-ee8362feea39 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.550244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.550913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.551185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.551450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.561990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.563447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.571234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017c2f1-f6ab-4738-bf21-f147f8b4791e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:26.578693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:26.579657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:26.580578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:26.580578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:26.580578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:26.584524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 438/870] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:26.585345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.598597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:26.617475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.618600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.619298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.619783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.621395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:26.621892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.623745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.644420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 433/871] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:26.646755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.699448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8b7a0-830c-4e50-9943-519238826d17 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.716703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:26.734752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.735390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.735671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.735933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.736598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:26.736893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.738364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.756853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 439/872] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:26.758155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:26.778651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34870d0-2e78-4577-87b7-274cf53a992e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.824530 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.825678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.826146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.826636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.827822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=01afe498af8c42e8bca4e993897ad1d9) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:26.828911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.830794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.831388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:26.844906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 434/873] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/role_assignments?user.id=01afe498af8c42e8bca4e993897ad1d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:26.875525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c224dbc-2f46-469f-b66a-dedddebedf6b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:26.913752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:26.914397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:26.914671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:26.914939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:26.923059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:26.924637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:26.937780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:26.938755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:26.939638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 440/874] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:26.944887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:27.058461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c729a8-7638-4826-9452-3256b4c0c4b6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:27.079950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:27.096495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9af041-8206-43a4-9eee-8eec587cd8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:27.097102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9af041-8206-43a4-9eee-8eec587cd8cb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:27.097378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9af041-8206-43a4-9eee-8eec587cd8cb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:27.097646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9af041-8206-43a4-9eee-8eec587cd8cb None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:27.098342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9af041-8206-43a4-9eee-8eec587cd8cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:27.098642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9af041-8206-43a4-9eee-8eec587cd8cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:27.100078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9af041-8206-43a4-9eee-8eec587cd8cb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:27.113553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 435/875] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:27.122101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:27.235392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc3df1-e292-4238-ad3b-a47e3f6a6b32 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:27.264641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:27.271460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1487c4-400d-4337-b096-eed05cec42a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:27.272065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1487c4-400d-4337-b096-eed05cec42a3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:27.272365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1487c4-400d-4337-b096-eed05cec42a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:27.272631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1487c4-400d-4337-b096-eed05cec42a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:27.273344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1487c4-400d-4337-b096-eed05cec42a3 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=e8008fa945f540a18aac7b0f81edf0d9) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:27.273705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1487c4-400d-4337-b096-eed05cec42a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:27.275066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1487c4-400d-4337-b096-eed05cec42a3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:27.281722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 441/876] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:27 2022] GET /identity/v3/role_assignments?user.id=e8008fa945f540a18aac7b0f81edf0d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:27.833771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194465aa-7510-43b9-915e-7a8d5a75a47f None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:27.834312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194465aa-7510-43b9-915e-7a8d5a75a47f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:27.834582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194465aa-7510-43b9-915e-7a8d5a75a47f None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:27.836346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 436/877] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:27.842029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e76028-049d-4331-a81d-7a2aa9482140 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:27.842311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e76028-049d-4331-a81d-7a2aa9482140 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:27.842580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e76028-049d-4331-a81d-7a2aa9482140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:27.862265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-98e76028-049d-4331-a81d-7a2aa9482140 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:27.863443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-98e76028-049d-4331-a81d-7a2aa9482140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:27.935863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 442/878] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:28.026180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee824bf-17e4-4729-8768-523839c6dae4 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.026494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee824bf-17e4-4729-8768-523839c6dae4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.026713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee824bf-17e4-4729-8768-523839c6dae4 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.028470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 437/879] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:28.033702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0a86-17d1-4b7c-808c-b031aba7c464 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.033988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0a86-17d1-4b7c-808c-b031aba7c464 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.034260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b0a86-17d1-4b7c-808c-b031aba7c464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.058522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-556b0a86-17d1-4b7c-808c-b031aba7c464 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:28.060072 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-556b0a86-17d1-4b7c-808c-b031aba7c464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:28.132439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 443/880] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:28.205960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee824bf-17e4-4729-8768-523839c6dae4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.308577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee824bf-17e4-4729-8768-523839c6dae4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:28.349064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:28.349703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.350037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.350307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.350952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:28.352366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:28.361869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:28.362811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:28.364076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 438/881] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:28.369212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b0a86-17d1-4b7c-808c-b031aba7c464 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.411500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.427960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:28.469853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:28.484755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b0a86-17d1-4b7c-808c-b031aba7c464 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:28.523131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:28.523738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.524009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.524285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.525164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:28.525464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:28.527070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:28.532608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 444/882] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:28.542849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.549649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462ec97-592a-48f9-bf8e-063c2bc1e0a6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:28.584024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:28.584675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.584946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.585201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.585778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:28.587147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:28.595886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:28.596913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:28.597771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:28.600506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 439/883] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:28.602991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.669191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7a44f-84de-41f9-a72d-99fd86df2b94 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:28.688716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:28.705924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a05cc-47fc-4032-9f9d-857f0b862dad None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:28.709286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:28.710130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.710601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.711089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.711915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:28.712442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:28.714198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:28.715381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:28.726629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 445/884] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:28.727920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.742431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:28.743013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.743286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.743554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.744188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:28.744512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:28.745938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:28.751230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 440/885] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:28.756743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.841168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7733-fbbb-4636-a709-fb1ee57c87aa None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:28.857153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472a5ba-edec-4e33-abc0-6404b3ca35a7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:28.878941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45973a13-508e-465b-9e33-0ca6989fe543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:28.879546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973a13-508e-465b-9e33-0ca6989fe543 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.879819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973a13-508e-465b-9e33-0ca6989fe543 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.880161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973a13-508e-465b-9e33-0ca6989fe543 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.880793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973a13-508e-465b-9e33-0ca6989fe543 None admin] RBAC: Adding query filter params (interface=public, service_id=42014dbc52dc4abf8a77a4f1f41a344d, region_id=RegionOne) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:28.881231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973a13-508e-465b-9e33-0ca6989fe543 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:28.882450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973a13-508e-465b-9e33-0ca6989fe543 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:28.892314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 446/886] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/endpoints?service_id=42014dbc52dc4abf8a77a4f1f41a344d&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:28.892314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1129c-fdc6-4143-ad03-71bc28fa5b4a None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.892314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1129c-fdc6-4143-ad03-71bc28fa5b4a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.892314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1129c-fdc6-4143-ad03-71bc28fa5b4a None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.892314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 447/887] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:28.895754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1129c-fdc6-4143-ad03-71bc28fa5b4a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:28.897192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe3bed-a28b-485f-8f0f-9ac2f5c6ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:28.897800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3bed-a28b-485f-8f0f-9ac2f5c6ce1d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.898072 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3bed-a28b-485f-8f0f-9ac2f5c6ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.898348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3bed-a28b-485f-8f0f-9ac2f5c6ce1d None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.898960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe3bed-a28b-485f-8f0f-9ac2f5c6ce1d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:28.899258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe3bed-a28b-485f-8f0f-9ac2f5c6ce1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:28.900653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe3bed-a28b-485f-8f0f-9ac2f5c6ce1d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:28.905830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 441/888] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:28.907051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6024b5-7d7b-40f2-8c4e-d84e994450aa None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:28.907327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6024b5-7d7b-40f2-8c4e-d84e994450aa None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:28.907590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6024b5-7d7b-40f2-8c4e-d84e994450aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:28.925619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-dd6024b5-7d7b-40f2-8c4e-d84e994450aa None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:28.926719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-dd6024b5-7d7b-40f2-8c4e-d84e994450aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:28.950609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:28.956953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:28.999877 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 442/889] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:29.000774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6024b5-7d7b-40f2-8c4e-d84e994450aa None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:29.019958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1129c-fdc6-4143-ad03-71bc28fa5b4a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:29.065871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:29.066651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.067025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.067295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.067897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:29.069936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:29.077239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 448/890] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:38:29.120687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6024b5-7d7b-40f2-8c4e-d84e994450aa None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:29.161091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:29.161962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.162369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.162809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.163715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] RBAC: Adding query filter params (interface=public, service_id=76f675fcc7814bc18161d890aef62329, region_id=RegionOne) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:29.164153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:29.165770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:29.174810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 443/891] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:38:28 2022] GET /identity/v3/endpoints?service_id=76f675fcc7814bc18161d890aef62329&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:29.181918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:29.282349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:29.290626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200ecd3-7493-4029-abd4-1cbef70c892b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:29.301481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:29.316934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:29.327603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09f0d-311d-40f3-b4dd-2028cbb90860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:29.328505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f0d-311d-40f3-b4dd-2028cbb90860 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.329086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f0d-311d-40f3-b4dd-2028cbb90860 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.329529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f0d-311d-40f3-b4dd-2028cbb90860 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.330303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09f0d-311d-40f3-b4dd-2028cbb90860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:29.331962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09f0d-311d-40f3-b4dd-2028cbb90860 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:29.339569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 449/892] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:29 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:38:29.360370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d81e7-c1a1-487b-9eb1-1d49c590abf3 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.360885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d81e7-c1a1-487b-9eb1-1d49c590abf3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.361306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d81e7-c1a1-487b-9eb1-1d49c590abf3 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.362970 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 450/893] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:29.368609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9be51-2883-46e4-874a-66c221b4526b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.368946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9be51-2883-46e4-874a-66c221b4526b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.369138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9be51-2883-46e4-874a-66c221b4526b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.389241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-99b9be51-2883-46e4-874a-66c221b4526b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:29.390447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-99b9be51-2883-46e4-874a-66c221b4526b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:29.410305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277b3fb-5907-45ba-b09c-2a129fba1645 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:29.465044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:29.466157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.466631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.467186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.467992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:29.469630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:29.476744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 451/894] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:29.480088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:29.481119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:29.481983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:29.484550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 444/895] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:29.492295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9be51-2883-46e4-874a-66c221b4526b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:29.593697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9be51-2883-46e4-874a-66c221b4526b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:29.596924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:29.630197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:29.630796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.631114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.631382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.632003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:29.632316 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:29.633676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:29.639040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 452/896] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:29.644654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:29.740109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:29.748554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746258-1d20-40a2-859c-5850b7140485 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:29.765844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:29.785867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:29.786856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.787323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.787821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.788684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:29.789262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:29.790832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:29.796534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 445/897] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:29.882265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4452d8e-6cee-45d9-b581-645716f2cfef None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:29.919633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:29.920307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:29.920590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:29.920859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:29.932705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:29.938753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:29.939728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:29.940609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 453/898] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Aug 12 10:38:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:29.948333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:30.053139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224088-d84d-4cdb-830d-663d112dd8f5 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:30.088557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:30.089216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:30.089548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:30.089820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:30.090486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:30.090786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:30.092171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:30.100803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 446/899] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Aug 12 10:38:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:30.106794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:30.140581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:30.216399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b8862-8102-4145-9a92-56a1b3fb36b5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:30.254458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:30.255103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:30.255385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:30.255651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:30.269019 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:30.270438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:30.284759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:30.285749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:30.286723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:30.286723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:30.286723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:30.286723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 454/900] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:30.293507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:30.316957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:30.399607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de6651-f6a6-479e-b2dc-1d4762dc2aae None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:30.434153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:30.434755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:30.435028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:30.435295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:30.435929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:30.436222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:30.437612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:30.450067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 447/901] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:30.455587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:30.543940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:30.565840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f1c16-4f06-4166-85c0-236a6cfbd753 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:30.600630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84299043-9094-471e-aed7-3eba927239fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:30.601185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84299043-9094-471e-aed7-3eba927239fe None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:30.601462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84299043-9094-471e-aed7-3eba927239fe None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:30.601733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84299043-9094-471e-aed7-3eba927239fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:30.609413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84299043-9094-471e-aed7-3eba927239fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:30.610854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84299043-9094-471e-aed7-3eba927239fe None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-84299043-9094-471e-aed7-3eba927239fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:30.621952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:30.622872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:30.624461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 455/902] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:38:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:30.628738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:30.711838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:30.731953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741147-f962-4fcd-a207-7496c5ca4c53 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:30.768021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:30.768654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:30.768927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:30.769196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:30.769929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:30.770231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:30.771729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:30.790870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 448/903] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Aug 12 10:38:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:30.797385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84299043-9094-471e-aed7-3eba927239fe None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:30.902572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84299043-9094-471e-aed7-3eba927239fe None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:30.937691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843cce6-20e8-4912-97a9-dd8d577a1579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:30.938293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843cce6-20e8-4912-97a9-dd8d577a1579 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:30.938565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843cce6-20e8-4912-97a9-dd8d577a1579 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:30.938831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843cce6-20e8-4912-97a9-dd8d577a1579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:30.950019 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:31.016227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843cce6-20e8-4912-97a9-dd8d577a1579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.018120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843cce6-20e8-4912-97a9-dd8d577a1579 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:31.116987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 456/904] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:30 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:31.122614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.148535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfad17a-6222-42b4-8dfb-df1d081711e0 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.149042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfad17a-6222-42b4-8dfb-df1d081711e0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.149449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfad17a-6222-42b4-8dfb-df1d081711e0 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.151091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 457/905] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:31.158181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908cac9-dd12-4fde-8f45-50698238581e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.158405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908cac9-dd12-4fde-8f45-50698238581e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.158683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908cac9-dd12-4fde-8f45-50698238581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.181253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b908cac9-dd12-4fde-8f45-50698238581e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:31.182712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b908cac9-dd12-4fde-8f45-50698238581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:31.220196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:31.233239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a9261-7abd-46e5-8624-e43c2308650f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.260401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 458/906] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:31.269793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba0a7-9b41-48a1-ae5a-ea1596907186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:31.270419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba0a7-9b41-48a1-ae5a-ea1596907186 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.270688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba0a7-9b41-48a1-ae5a-ea1596907186 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.270956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba0a7-9b41-48a1-ae5a-ea1596907186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.277823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:31.281603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dba0a7-9b41-48a1-ae5a-ea1596907186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.283112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dba0a7-9b41-48a1-ae5a-ea1596907186 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-37dba0a7-9b41-48a1-ae5a-ea1596907186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:31.298213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:31.299193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:31.300694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:31.300694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:31.300694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:31.300694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 449/907] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:31.305192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908cac9-dd12-4fde-8f45-50698238581e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.361317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b847df-ca43-4d9d-a906-11a68a9448c8 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.361683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b847df-ca43-4d9d-a906-11a68a9448c8 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.361848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b847df-ca43-4d9d-a906-11a68a9448c8 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.363437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 450/908] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:31.368339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81f3eb-3179-4856-ad4d-a1112fa18ab9 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.368620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81f3eb-3179-4856-ad4d-a1112fa18ab9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.368892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81f3eb-3179-4856-ad4d-a1112fa18ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.389701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ce81f3eb-3179-4856-ad4d-a1112fa18ab9 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:31.390891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ce81f3eb-3179-4856-ad4d-a1112fa18ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:31.417668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908cac9-dd12-4fde-8f45-50698238581e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.460832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:31.461497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.461736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.461954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.462598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:31.462894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.464329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:31.464775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 451/909] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:31.480987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 459/910] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:31.485377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81f3eb-3179-4856-ad4d-a1112fa18ab9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.539558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.572826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:31.598952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81f3eb-3179-4856-ad4d-a1112fa18ab9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.636930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:31.643011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:31.643908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.644190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.644492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.649475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175a062-5c63-4716-b7d6-4102628b3ffb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.652709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.656389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:31.668583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:31.669484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:31.670386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:31.673028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 452/911] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:31.679071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.692847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:31.693477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.693751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.694019 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.694598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.696665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:31.707497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:31.708497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:31.709385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 460/912] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:31.716893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.800497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc286ad0-ae4c-4d9b-9560-271257ef9464 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.820078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:31.828300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e160064-e017-42bb-8a7f-2c89bc3c7ee1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.837561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:31.838167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.838435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.839041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.839768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:31.840067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.841978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:31.854067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 453/913] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:31.854896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.870577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:31.871177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.871449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.871729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.872375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:31.872675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.874068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:31.878871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 461/914] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:31.879458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:31.898689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:31.953303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62c5f-fd2c-458d-98ba-12b167aee074 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.977688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c03875-fc36-464f-8b16-5cfc700abe0b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:31.988123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a18bc-948c-4ca9-b511-f33d33f84df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:31.988785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a18bc-948c-4ca9-b511-f33d33f84df1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:31.989059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a18bc-948c-4ca9-b511-f33d33f84df1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:31.989326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a18bc-948c-4ca9-b511-f33d33f84df1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:31.989908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a18bc-948c-4ca9-b511-f33d33f84df1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:31.991286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a18bc-948c-4ca9-b511-f33d33f84df1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-ab5a18bc-948c-4ca9-b511-f33d33f84df1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:32.000682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:32.001750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:32.002590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 454/915] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:32.003422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd1ac-8c2a-49bf-a16b-5caa23fa98ba None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.003422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd1ac-8c2a-49bf-a16b-5caa23fa98ba None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.003542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd1ac-8c2a-49bf-a16b-5caa23fa98ba None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.004672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 455/916] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:32.005349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafd1ac-8c2a-49bf-a16b-5caa23fa98ba None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:32.016765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:32.017373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.017644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.017909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.018633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] RBAC: Adding query filter params (role.id=00b7ebef18e042439e2d5c6779cfae59, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=e8008fa945f540a18aac7b0f81edf0d9) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:32.018936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:32.020386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.025626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 462/917] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:31 2022] GET /identity/v3/role_assignments?user.id=e8008fa945f540a18aac7b0f81edf0d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=00b7ebef18e042439e2d5c6779cfae59 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:32.026467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:32.056466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:32.133987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafd1ac-8c2a-49bf-a16b-5caa23fa98ba None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:32.137840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:32.143997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c359f3-e675-410c-aaee-d940d3bd5cfb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:32.181314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5536e-d7a4-4123-8025-98b0d06e912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:32.182047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5536e-d7a4-4123-8025-98b0d06e912c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.182378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5536e-d7a4-4123-8025-98b0d06e912c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.182703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5536e-d7a4-4123-8025-98b0d06e912c None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.183773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5536e-d7a4-4123-8025-98b0d06e912c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:32.184120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5536e-d7a4-4123-8025-98b0d06e912c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:32.185444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:32.185628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5536e-d7a4-4123-8025-98b0d06e912c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.186363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.186897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.187304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.188196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:32.188647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:32.190137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.191075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 456/918] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:32.192241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a05c0-a729-4b5b-81eb-e5a4c4c36a0d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.192545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a05c0-a729-4b5b-81eb-e5a4c4c36a0d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.192812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a05c0-a729-4b5b-81eb-e5a4c4c36a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.200751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 463/919] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:32.201714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:32.216583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-5f4a05c0-a729-4b5b-81eb-e5a4c4c36a0d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:32.217681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5f4a05c0-a729-4b5b-81eb-e5a4c4c36a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:32.288823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 457/920] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:32.289664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a05c0-a729-4b5b-81eb-e5a4c4c36a0d None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:32.305850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:32.326398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e544bc3-d7aa-46c6-a4ed-10fdcc4f2931 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:32.375238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:32.375842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.376111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.379088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.379973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:32.382031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.401691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:32.406254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a05c0-a729-4b5b-81eb-e5a4c4c36a0d None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:32.412608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 464/921] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:32.456539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:32.457488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.457935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.458369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.459171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:32.459633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:32.461260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.467800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 458/922] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Aug 12 10:38:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:32.474143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:32.566998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:32.595703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440bfac-993e-4a33-85ae-84d6fad400a3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:32.647780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a8ec6-9aee-41b0-a7e3-a79e9440dc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:32.648484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a8ec6-9aee-41b0-a7e3-a79e9440dc45 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.648760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a8ec6-9aee-41b0-a7e3-a79e9440dc45 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.649025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a8ec6-9aee-41b0-a7e3-a79e9440dc45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.649699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a8ec6-9aee-41b0-a7e3-a79e9440dc45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:32.651759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a8ec6-9aee-41b0-a7e3-a79e9440dc45 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.673066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:32.682698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 465/923] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:32.696628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba9b2b-f489-4584-a9d5-6f7daabd66a3 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:32.747351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac0ce7-f4d8-4138-bc87-66d01961ffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:32.748484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac0ce7-f4d8-4138-bc87-66d01961ffa6 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:32.749034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac0ce7-f4d8-4138-bc87-66d01961ffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:32.749506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac0ce7-f4d8-4138-bc87-66d01961ffa6 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:32.750373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac0ce7-f4d8-4138-bc87-66d01961ffa6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:32.752044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac0ce7-f4d8-4138-bc87-66d01961ffa6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:32.771728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 459/924] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:32 2022] POST /identity/v3/services => generated 241 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:38:33.948554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de2b13-4b23-495a-8adc-89960a6b422b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:33.949773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de2b13-4b23-495a-8adc-89960a6b422b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:33.949773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de2b13-4b23-495a-8adc-89960a6b422b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:33.950694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 466/925] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:33.956019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab3668-380c-418f-adbe-86ad67301962 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:33.956454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab3668-380c-418f-adbe-86ad67301962 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:33.956631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab3668-380c-418f-adbe-86ad67301962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:33.977721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-75ab3668-380c-418f-adbe-86ad67301962 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:33.979006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-75ab3668-380c-418f-adbe-86ad67301962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:33.988904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:34.061722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 460/926] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:34.324417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de2b13-4b23-495a-8adc-89960a6b422b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:34.368649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71767469-a16d-4122-b993-4a60ae2d4348 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.368986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71767469-a16d-4122-b993-4a60ae2d4348 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.369186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71767469-a16d-4122-b993-4a60ae2d4348 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.371019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 461/927] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:34.376367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d111c4-49f5-4d6b-967e-b79431be80e7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.376645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d111c4-49f5-4d6b-967e-b79431be80e7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.376912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d111c4-49f5-4d6b-967e-b79431be80e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.396182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b0d111c4-49f5-4d6b-967e-b79431be80e7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:34.397394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b0d111c4-49f5-4d6b-967e-b79431be80e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:34.438433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de2b13-4b23-495a-8adc-89960a6b422b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:34.480675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 462/928] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:34.494311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:34.494944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.495218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.495675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.506555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:34.511520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:34.512538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:34.513440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 467/929] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Aug 12 10:38:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:34.523933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d111c4-49f5-4d6b-967e-b79431be80e7 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:34.601114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:34.633781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d111c4-49f5-4d6b-967e-b79431be80e7 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:34.673133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed7423-6a14-49b5-96d1-87551a25054c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:34.673740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed7423-6a14-49b5-96d1-87551a25054c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.674012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed7423-6a14-49b5-96d1-87551a25054c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.674276 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed7423-6a14-49b5-96d1-87551a25054c None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.674951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed7423-6a14-49b5-96d1-87551a25054c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:34.675249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed7423-6a14-49b5-96d1-87551a25054c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:34.676652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed7423-6a14-49b5-96d1-87551a25054c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:34.685696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 463/930] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Aug 12 10:38:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:34.693350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:34.701017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2552-636a-48cc-9537-703effa074fe None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.701297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2552-636a-48cc-9537-703effa074fe None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.701572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2552-636a-48cc-9537-703effa074fe None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.703142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 464/931] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:34.709765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f13c0-d9fa-464e-b040-04715c9077c5 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.710044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f13c0-d9fa-464e-b040-04715c9077c5 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.710565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f13c0-d9fa-464e-b040-04715c9077c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.734422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-595f13c0-d9fa-464e-b040-04715c9077c5 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:34.735610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-595f13c0-d9fa-464e-b040-04715c9077c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:34.757625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:34.802932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b25455-2c1e-4497-8bf1-82124d0d3c49 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:34.807494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 465/932] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:34.808524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f13c0-d9fa-464e-b040-04715c9077c5 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:34.838303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9921e5-479b-46fe-bdf3-b8cf456c1b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:34.838924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9921e5-479b-46fe-bdf3-b8cf456c1b0a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.839361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9921e5-479b-46fe-bdf3-b8cf456c1b0a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.839615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9921e5-479b-46fe-bdf3-b8cf456c1b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.849693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9921e5-479b-46fe-bdf3-b8cf456c1b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:34.851442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9921e5-479b-46fe-bdf3-b8cf456c1b0a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-da9921e5-479b-46fe-bdf3-b8cf456c1b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:34.866979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:34.868057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:34.869199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:34.869199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:34.869199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:34.872976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 468/933] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:34.874208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5a1d-e72e-46a8-be8e-e969f4be6be7 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.874492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5a1d-e72e-46a8-be8e-e969f4be6be7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.874863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:34.875429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f5a1d-e72e-46a8-be8e-e969f4be6be7 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.876994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 469/934] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:34.877489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5a1d-e72e-46a8-be8e-e969f4be6be7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:34.914637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f13c0-d9fa-464e-b040-04715c9077c5 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:34.949680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4630ce-ffd5-44c5-b9bf-30532b5abe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:34.950305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4630ce-ffd5-44c5-b9bf-30532b5abe0c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.950576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4630ce-ffd5-44c5-b9bf-30532b5abe0c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.950838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4630ce-ffd5-44c5-b9bf-30532b5abe0c None admin] PATH_INFO: `/v3/services/compute` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.951410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4630ce-ffd5-44c5-b9bf-30532b5abe0c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:34.952814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4630ce-ffd5-44c5-b9bf-30532b5abe0c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-3d4630ce-ffd5-44c5-b9bf-30532b5abe0c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:34.961379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:34.962937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:34.963790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 466/935] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:34.964947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32658540-5cd4-49c7-a13a-be1aea6a03b1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:34.964947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32658540-5cd4-49c7-a13a-be1aea6a03b1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:34.964947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32658540-5cd4-49c7-a13a-be1aea6a03b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:34.981874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-32658540-5cd4-49c7-a13a-be1aea6a03b1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:34.983049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-32658540-5cd4-49c7-a13a-be1aea6a03b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:34.986890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f5a1d-e72e-46a8-be8e-e969f4be6be7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:34.999851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.025474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.026098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.026371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.026679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.027407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:35.027710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.029099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.040769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 470/936] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:35.041474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.064302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 467/937] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:35.064302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32658540-5cd4-49c7-a13a-be1aea6a03b1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.119492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.154228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b34a9-d3c5-46cb-b45a-df20ae16d0d7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.169072 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32658540-5cd4-49c7-a13a-be1aea6a03b1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.188753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.189349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.189620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.189883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.190498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] RBAC: Adding query filter params (name=compute) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:35.190799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.192149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.196904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 471/938] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:35.197750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.211541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.212403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.212676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.212978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.220237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.221753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:35.233187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:35.234166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:35.235128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:35.235736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.235810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 468/939] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:35.235810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.306750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910f6ab-cdf6-4db6-83d1-dcc948368cbc None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.340454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.341069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.341338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.341600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.341698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2088c-9868-44fc-a18a-2bd5b9d8e097 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.342226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.343634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.346266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING keystone.server.flask.common [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 12 10:38:35.357894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.363406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.sql.core [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 12 10:38:35.363406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 12 10:38:35.363406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [parameters: {'id': '2103f485aa204638b7245f8668765137', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 12 10:38:35.363406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93139) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 12 10:38:35.375304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.375839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.376127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.376436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.377062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:35.377359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.378717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2103f485aa204638b7245f8668765137.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2103f485aa204638b7245f8668765137. Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application conn.query(q) Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 10:38:35.379214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application result.read() Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 12 10:38:35.380084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application next(self.gen) Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application yield resource Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application next(self.gen) Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self.session.rollback() Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self.force_reraise() Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise self.value Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application session.commit() Aug 12 10:38:35.386913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._prepare_impl() Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self.session.flush() Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._flush(objects) Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application compat.raise_( Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application flush_context.execute() Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 12 10:38:35.387885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rec.execute(self) Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 12 10:38:35.388842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application conn.query(q) Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application result.read() Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 10:38:35.389630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [parameters: {'id': '2103f485aa204638b7245f8668765137', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.390553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:35.391366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2103f485aa204638b7245f8668765137. Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 472/940] 10.222.0.23 () {60 vars in 1140 bytes} [Fri Aug 12 10:38:35 2022] POST /identity/v3/projects => generated 262 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:35.393584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.395029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 469/941] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:35.395029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.504502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.505952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bd45-6c2c-4d2a-8ec7-da39bdab8732 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.518139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e78de-eefe-47a6-a82b-20bdb20b0aa9 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.547285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.547285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.547285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.547285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.547721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.547991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.559558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.560159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.560452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.560716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:35.561184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:35.562187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:35.563035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:35.563774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:35.563774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.563907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 470/942] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:35.563907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.564037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.585099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 473/943] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:35.586002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.613588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.669761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c3ef9-b86b-4ec4-a759-fd67dd1b07be None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.694634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c91a7-dc2f-425b-8c8d-17d93a1d428b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.703037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.703660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.703937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.704195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.710805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.712246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:35.723542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:35.724514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:35.725399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:35.729558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 471/944] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:35.730629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.730746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.731307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.731578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.731841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.732490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:35.732783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.734146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.740489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 474/945] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:35.740489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.756515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.832344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5216f-cc4a-4c11-acb2-ab1250d8a8af None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.834678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:35.850627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258c3b-bfc8-4a76-b6cc-795a9a1b5bca None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:35.865853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.866471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.866741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.867003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.877527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.878995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.884724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:35.885379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:35.885648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:35.885945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:35.886941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:35.887214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:35.888638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:35.893684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:35.894605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:35.900685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:35.900685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:35.900685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:35.900685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 472/946] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:35.900685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.908899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 475/947] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:35.909575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:35.992774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:36.017156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca72c1-8a3e-4fe2-99e8-135602c60214 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:36.030844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec7fd9-800d-43e2-8f24-c07c6d8eb891 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:36.053450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:36.054057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:36.054326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:36.054591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:36.055219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:36.055512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:36.056924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:36.062570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 473/948] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:36.069397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:36.076130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:36.077005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:36.077433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:36.077853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:36.078790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:36.079094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:36.080484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:36.093998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:36.096080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 476/949] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:36.102296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:36.190510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a880284-d5c4-4807-8e00-a074b6482e9d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:36.225001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:36.225969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:36.226407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:36.226814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:36.227755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] RBAC: Adding query filter params (interface=public, service_id=a3670d6f12ed4f49b3c75e478dc83236, region_id=RegionOne) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:36.228391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:36.229961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:36.231277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736dfbb-9c8b-4895-95cd-dfa4631f1e3f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:36.240721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 474/950] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:38:36 2022] GET /identity/v3/endpoints?service_id=a3670d6f12ed4f49b3c75e478dc83236&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:36.242303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:36.267892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:36.275494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:36.276699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:36.277054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:36.277512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:36.286822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:36.288447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:36.304179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:36.305133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:36.306274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 477/951] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:36.311264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:36.352721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:36.366919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43507ed9-4212-44bc-9c1f-e7f690890077 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:36.410596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:36.411247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:36.411519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:36.411789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:36.412496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:36.414304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:36.428434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 475/952] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:36 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:38:36.441786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8d734-5ee2-46ac-b636-ad10478d54c9 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:36.481921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76a6c5-58b9-4967-b1fb-ff4fba30c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:36.482599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a6c5-58b9-4967-b1fb-ff4fba30c899 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:36.482917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a6c5-58b9-4967-b1fb-ff4fba30c899 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:36.483186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a6c5-58b9-4967-b1fb-ff4fba30c899 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:36.483907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76a6c5-58b9-4967-b1fb-ff4fba30c899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:36.484203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76a6c5-58b9-4967-b1fb-ff4fba30c899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:36.485704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76a6c5-58b9-4967-b1fb-ff4fba30c899 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:36.500915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 478/953] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:36.509489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:36.619779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68401493-d8a8-4fe7-bab3-520bedf9aa0b None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:36.655450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0323b-6ab9-4061-af86-eec3b7699f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:36.656610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0323b-6ab9-4061-af86-eec3b7699f73 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:36.657107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0323b-6ab9-4061-af86-eec3b7699f73 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:36.657592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0323b-6ab9-4061-af86-eec3b7699f73 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/e8008fa945f540a18aac7b0f81edf0d9/roles/00b7ebef18e042439e2d5c6779cfae59` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:36.659916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0323b-6ab9-4061-af86-eec3b7699f73 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=e8008fa945f540a18aac7b0f81edf0d9, role_id=00b7ebef18e042439e2d5c6779cfae59)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:36.661597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0323b-6ab9-4061-af86-eec3b7699f73 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:36.680310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 476/954] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:38:36 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/e8008fa945f540a18aac7b0f81edf0d9/roles/00b7ebef18e042439e2d5c6779cfae59 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:38:37.364363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6154a-db64-4e9c-a3e5-734bdf3240d5 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:37.364897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6154a-db64-4e9c-a3e5-734bdf3240d5 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:37.365149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6154a-db64-4e9c-a3e5-734bdf3240d5 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:37.367468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 479/955] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:37.375712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67220d0e-cf27-49a4-8c16-445fd03a80ed None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:37.376237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67220d0e-cf27-49a4-8c16-445fd03a80ed None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:37.376656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67220d0e-cf27-49a4-8c16-445fd03a80ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:37.399273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-67220d0e-cf27-49a4-8c16-445fd03a80ed None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:37.400489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-67220d0e-cf27-49a4-8c16-445fd03a80ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:37.402829 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:37.471239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 477/956] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:37.735730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6154a-db64-4e9c-a3e5-734bdf3240d5 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:37.793738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:37.843934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6154a-db64-4e9c-a3e5-734bdf3240d5 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:37.878209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33fb7-dea9-4be7-9841-06aa6e093b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:37.878812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33fb7-dea9-4be7-9841-06aa6e093b2e None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:37.879084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33fb7-dea9-4be7-9841-06aa6e093b2e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:37.879353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33fb7-dea9-4be7-9841-06aa6e093b2e None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:37.880006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33fb7-dea9-4be7-9841-06aa6e093b2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:37.881458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33fb7-dea9-4be7-9841-06aa6e093b2e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:37.900954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 480/957] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:37 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:37.971738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb6ea-243a-4dda-8723-a43d0b0e4034 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:37.972075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb6ea-243a-4dda-8723-a43d0b0e4034 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:37.972312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb6ea-243a-4dda-8723-a43d0b0e4034 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:37.973990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 478/958] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:37.979662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838f0cd-c7e7-40eb-ad3a-d72220c7db32 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:37.980179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838f0cd-c7e7-40eb-ad3a-d72220c7db32 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:37.980929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838f0cd-c7e7-40eb-ad3a-d72220c7db32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.001399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9838f0cd-c7e7-40eb-ad3a-d72220c7db32 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:38.003155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9838f0cd-c7e7-40eb-ad3a-d72220c7db32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:38.084016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 481/959] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:38.357293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb6ea-243a-4dda-8723-a43d0b0e4034 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:38.362367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec2fb7-3626-4043-9e98-163f457b6f50 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.362649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec2fb7-3626-4043-9e98-163f457b6f50 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.362923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec2fb7-3626-4043-9e98-163f457b6f50 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.364581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 482/960] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:38.370065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fd7d-046c-4853-973c-4b0a9397d3ee None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.370416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fd7d-046c-4853-973c-4b0a9397d3ee None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.370686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787fd7d-046c-4853-973c-4b0a9397d3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.388957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b787fd7d-046c-4853-973c-4b0a9397d3ee None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:38.390121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b787fd7d-046c-4853-973c-4b0a9397d3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:38.423008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:38.473870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 483/961] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:38.488623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb6ea-243a-4dda-8723-a43d0b0e4034 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:38.526948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:38.527539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.527816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.528086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.528765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:38.530209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:38.532780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING keystone.server.flask.common [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 12 10:38:38.542169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:38.547903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.sql.core [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 12 10:38:38.547903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 12 10:38:38.547903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [parameters: {'user_id': '2f14e140313f432e8ef97a42f5b392ce', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 12 10:38:38.547903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93138) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c841b578-d2cb-4a80-9a83-b678a8952f25 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application conn.query(q) Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result.read() Aug 12 10:38:38.554649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 10:38:38.555586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application next(self.gen) Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application yield resource Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application next(self.gen) Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.session.rollback() Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.force_reraise() Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise self.value Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application session.commit() Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 12 10:38:38.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._prepare_impl() Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.session.flush() Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._flush(objects) Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application compat.raise_( Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application flush_context.execute() Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rec.execute(self) Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 12 10:38:38.558197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 10:38:38.559229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application conn.query(q) Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application result.read() Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 10:38:38.560175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [parameters: {'user_id': '2f14e140313f432e8ef97a42f5b392ce', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:38.562206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 12 10:38:38.563940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 12 10:38:38.572955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 12 10:38:38.572955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 12 10:38:38.572955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 12 10:38:38.572955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 12 10:38:38.572955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 12 10:38:38.572955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.572955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 479/962] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Aug 12 10:38:38 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:38.573947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787fd7d-046c-4853-973c-4b0a9397d3ee None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:38.651806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1ab0-1ac7-4191-881d-d990251a775c None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.652122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1ab0-1ac7-4191-881d-d990251a775c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.652495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e1ab0-1ac7-4191-881d-d990251a775c None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.661064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 480/963] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:38.662300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270d580-66ce-44a0-be11-8ae247a40197 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.662577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270d580-66ce-44a0-be11-8ae247a40197 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.662843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270d580-66ce-44a0-be11-8ae247a40197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.675717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787fd7d-046c-4853-973c-4b0a9397d3ee None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:38.681538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a270d580-66ce-44a0-be11-8ae247a40197 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:38.683168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a270d580-66ce-44a0-be11-8ae247a40197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:38.721752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:38.722387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.722655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.722919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.733131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:38.735129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:38.751792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:38.753598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:38.754878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 484/964] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:38.755849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 481/965] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:38.762161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:38.769264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270d580-66ce-44a0-be11-8ae247a40197 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:38.804030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:38.877775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270d580-66ce-44a0-be11-8ae247a40197 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:38.904735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1274ae-43b6-441d-a5d0-9158c3c43cda None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:38.909049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:38.915995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:38.916846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.917278 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.917733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.918516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:38.920140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:38.929204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:38.930460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:38.931488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:38.938589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 482/966] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:38.940048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:38.950247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:38.950864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:38.951139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:38.951408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:38.952113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:38.952436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:38.954778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.007675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 485/967] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Aug 12 10:38:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:39.037416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.068675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b668d-7f8f-495f-b98f-b1aa9c546dc4 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.111747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.112814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.113324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.113858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.114722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:39.115250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:39.116950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.123598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 483/968] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:39.137619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.151181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:39.169005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea768d-173a-4299-a682-2ce93420d42a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.208749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:39.212982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.213624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.214003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.214540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.231537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:39.237143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:39.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:39.238956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 486/969] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Aug 12 10:38:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:39.247108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.261880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fead87-e8bd-47f4-ae3e-62ad0599b4da None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.302488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.303362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.303780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.304189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.304994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:39.306124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:39.307761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.314766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 484/970] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:39.321028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.366566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c7846-778c-49cd-81b4-c3cb3220bdbf None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.404427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.405047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.405321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.405579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.406241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:39.406534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:39.407917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.416365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 487/971] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Aug 12 10:38:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:39.422543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.426580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d2f4a-9902-4d3e-a435-c0504d6a5b50 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.434232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:39.452144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:39.464812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.465109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.465379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.465649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.466317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:39.467706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.488699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 485/972] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:39 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:38:39.536469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921e09b-0a09-41c2-ab0e-c5d4c474a409 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.581399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.582034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.582310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.582578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.594850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:39.596381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:39.612235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:39.613313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:39.619503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:39.619503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:39.619503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:39.619503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 488/973] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:39.621728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.731531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b173e-04d8-4647-ae44-99b1a264a974 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.765759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209328ce-efc8-4e44-ba16-0a9c46b23696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.766373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209328ce-efc8-4e44-ba16-0a9c46b23696 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.766632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209328ce-efc8-4e44-ba16-0a9c46b23696 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.766898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209328ce-efc8-4e44-ba16-0a9c46b23696 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.767545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209328ce-efc8-4e44-ba16-0a9c46b23696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:39.767841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209328ce-efc8-4e44-ba16-0a9c46b23696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:39.769248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209328ce-efc8-4e44-ba16-0a9c46b23696 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.781496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 486/974] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:39.787030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.846204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:39.892147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68b17c-37d9-47f6-a3c3-8449728698f8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:39.899788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1473-b81a-4bc5-8765-42406072ac13 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.900551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1473-b81a-4bc5-8765-42406072ac13 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.900831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1473-b81a-4bc5-8765-42406072ac13 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.902707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 487/975] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:39.908092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7740e-9cf8-4ccd-96a7-c0ac473bc6fd None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.908568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7740e-9cf8-4ccd-96a7-c0ac473bc6fd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.908839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7740e-9cf8-4ccd-96a7-c0ac473bc6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.927043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-25f7740e-9cf8-4ccd-96a7-c0ac473bc6fd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:39.928342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-25f7740e-9cf8-4ccd-96a7-c0ac473bc6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:39.928944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:39.929585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:39.929862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:39.930134 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:39.937979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:39.939439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:39.952217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:39.953245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:39.954847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 489/976] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:38:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:39.959021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:39.964779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.014402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 488/977] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:40.059658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d557826-6b82-4271-a8fe-74f342a8f9c0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.075646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.094003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.094722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.095094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.095472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.096297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:40.096707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.098232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.115864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 490/978] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Aug 12 10:38:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:40.121773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7740e-9cf8-4ccd-96a7-c0ac473bc6fd None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.220900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7740e-9cf8-4ccd-96a7-c0ac473bc6fd None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.253535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.254131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.254401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.254665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.266143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.290204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.325986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.327734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.375510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d12c79-df4e-428a-94d7-492ae4cd8a8c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.410626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.411245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.411516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.411783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.412366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.413716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:40.422413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:40.423397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:40.424242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 491/979] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:40.429225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.442444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 489/980] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:40.444384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.480819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.523045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.546740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce7f3-f300-4e41-af07-05bbc104d0d0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.558166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf1c2-a5f0-43f1-9e2d-d72fea8796c9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.583457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.584120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.584430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.584789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.585328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:40.585625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.587003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.591920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 492/981] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:40.597566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.599423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.600219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.600536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.601638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.613096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.614543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:40.629319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:40.630283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:40.631174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:40.631174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:40.631174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:40.635190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 490/982] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:40.636376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.700745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8addf0f-d61a-40fa-a00b-09276d173007 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.702351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.741000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1501e-4228-43c5-88fa-2e5abb3936c3 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.744774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.746227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.746706 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.747198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.748473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:40.748989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.750940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.760906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.761840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 493/983] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:40.766742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.779437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.779967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.780239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.780546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.781187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:40.781613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.782850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.796724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 491/984] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:40.802107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.876207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677f05d-23bd-4a67-b103-3b8025eda487 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.907644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeec47-6b53-4641-b462-a526ac60ccf8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:40.910956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.911568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.911842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.912108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.912835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] RBAC: Adding query filter params (interface=public, service_id=11854cbfd28f4d0a85ab7fcf483b8c60, region_id=RegionOne) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:40.913140 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.914528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.920290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 494/985] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/endpoints?service_id=11854cbfd28f4d0a85ab7fcf483b8c60&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:40.925622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:40.945171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:40.945792 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:40.946064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:40.946329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:40.953225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:40.954671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:40.957379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:40.965700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:40.966634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 492/986] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:40.973116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:41.016681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:41.038065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e43b4-466f-4760-9ef9-820592b802d9 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:41.074545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:41.075155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:41.075428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:41.075826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:41.076484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:41.077900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:41.079245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f2ab-63a5-4bf7-b8d7-b72b5e686dc2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:41.084602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 495/987] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:38:41.125897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c10179-3800-4ece-b547-10dd62d1afbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:41.126507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c10179-3800-4ece-b547-10dd62d1afbd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:41.126780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c10179-3800-4ece-b547-10dd62d1afbd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:41.127042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c10179-3800-4ece-b547-10dd62d1afbd None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:41.127744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c10179-3800-4ece-b547-10dd62d1afbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:41.128041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c10179-3800-4ece-b547-10dd62d1afbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:41.129538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c10179-3800-4ece-b547-10dd62d1afbd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:41.144587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 493/988] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:41.150577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:41.253243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:41.261581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8a2a-61e5-47ec-a415-4de6451b5152 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:41.296416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf7e2e-d972-4fca-8bcd-0be374808a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:41.296936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7e2e-d972-4fca-8bcd-0be374808a10 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:41.297208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7e2e-d972-4fca-8bcd-0be374808a10 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:41.297473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7e2e-d972-4fca-8bcd-0be374808a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:41.298192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf7e2e-d972-4fca-8bcd-0be374808a10 None admin] RBAC: Adding query filter params (role.id=00b7ebef18e042439e2d5c6779cfae59, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=e8008fa945f540a18aac7b0f81edf0d9) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:41.298494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf7e2e-d972-4fca-8bcd-0be374808a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:41.299916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf7e2e-d972-4fca-8bcd-0be374808a10 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:41.306047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 496/989] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:41 2022] GET /identity/v3/role_assignments?user.id=e8008fa945f540a18aac7b0f81edf0d9&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=00b7ebef18e042439e2d5c6779cfae59 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:41.392655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a26a-16d9-40e5-9e25-3597309baf25 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:41.392922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a26a-16d9-40e5-9e25-3597309baf25 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:41.393187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a26a-16d9-40e5-9e25-3597309baf25 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:41.394832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 494/990] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:41.400553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d4d4d-901e-43b4-8084-5a90998ad136 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:41.401076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d4d4d-901e-43b4-8084-5a90998ad136 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:41.401076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d4d4d-901e-43b4-8084-5a90998ad136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:41.425934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d13d4d4d-901e-43b4-8084-5a90998ad136 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:41.427149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d13d4d4d-901e-43b4-8084-5a90998ad136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:41.502396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 497/991] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:41.771964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3a26a-16d9-40e5-9e25-3597309baf25 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:41.873261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:41.877812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3a26a-16d9-40e5-9e25-3597309baf25 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:41.911663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:41.912287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:41.912610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:41.912875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:41.913443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:41.914793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:41.923452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:41.924444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:41.925315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 495/992] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:41.930695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d4d4d-901e-43b4-8084-5a90998ad136 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:42.020093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:42.033791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d4d4d-901e-43b4-8084-5a90998ad136 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:42.067728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:42.068359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:42.068633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:42.068894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:42.069511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:42.069801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:42.071161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:42.076509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 498/993] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:42.081995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:42.183481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd1d06-0d6e-4bd7-82e4-f74babc0ee8a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:42.217804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:42.218409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:42.218678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:42.218941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:42.219556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:42.219844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:42.221238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:42.226208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 496/994] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:42.232232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:42.330304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a854aae-0a32-40cc-a31f-5e25ffda6d92 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:42.373614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cbb0d-312e-4637-b51e-d6c1d6235cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:42.374212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cbb0d-312e-4637-b51e-d6c1d6235cbe None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:42.374481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cbb0d-312e-4637-b51e-d6c1d6235cbe None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:42.374739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cbb0d-312e-4637-b51e-d6c1d6235cbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:42.375428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cbb0d-312e-4637-b51e-d6c1d6235cbe None admin] RBAC: Adding query filter params (interface=admin, service_id=a3670d6f12ed4f49b3c75e478dc83236, region_id=RegionOne) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:42.375726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cbb0d-312e-4637-b51e-d6c1d6235cbe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:42.377117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cbb0d-312e-4637-b51e-d6c1d6235cbe None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:42.387254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 499/995] 10.222.0.23 () {56 vars in 1249 bytes} [Fri Aug 12 10:38:42 2022] GET /identity/v3/endpoints?service_id=a3670d6f12ed4f49b3c75e478dc83236&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:42.388879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:42.419253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:42.496192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735a4a4-63e2-4b15-a12b-5a7f3da6b804 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:42.534928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5f529-2a8e-40a3-ad8b-5079a022fcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:42.535520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5f529-2a8e-40a3-ad8b-5079a022fcc4 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:42.535788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5f529-2a8e-40a3-ad8b-5079a022fcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:42.536038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5f529-2a8e-40a3-ad8b-5079a022fcc4 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:42.536677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5f529-2a8e-40a3-ad8b-5079a022fcc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:42.538065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5f529-2a8e-40a3-ad8b-5079a022fcc4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:42.544517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 497/996] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:42 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:38:43.007885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b8b0d-3ae7-40e0-b366-f28d247b043b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.008481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b8b0d-3ae7-40e0-b366-f28d247b043b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.008577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b8b0d-3ae7-40e0-b366-f28d247b043b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.009958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 500/997] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:43.015739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79075e-b03e-4868-bf69-7b5706557256 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.016194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79075e-b03e-4868-bf69-7b5706557256 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.016346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79075e-b03e-4868-bf69-7b5706557256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.034435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ca79075e-b03e-4868-bf69-7b5706557256 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:43.035614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ca79075e-b03e-4868-bf69-7b5706557256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:43.106385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 498/998] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:43.301010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa623ca9-8414-4067-a5ba-23a46fe01032 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.301224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa623ca9-8414-4067-a5ba-23a46fe01032 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.301522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa623ca9-8414-4067-a5ba-23a46fe01032 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.302926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 501/999] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:43.307968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7c4d9-ad04-4826-a878-022193d6bc4c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.308389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7c4d9-ad04-4826-a878-022193d6bc4c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.308664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7c4d9-ad04-4826-a878-022193d6bc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.326689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4bf7c4d9-ad04-4826-a878-022193d6bc4c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:43.327839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4bf7c4d9-ad04-4826-a878-022193d6bc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:43.346143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:43.372723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa623ca9-8414-4067-a5ba-23a46fe01032 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:43.395616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:43.399000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 499/1000] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:43.484960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa623ca9-8414-4067-a5ba-23a46fe01032 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:43.521994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:43.522622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.522897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.523159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.523727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:43.525131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:43.534636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:43.535631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:43.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 502/1001] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:43.545602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7c4d9-ad04-4826-a878-022193d6bc4c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:43.661388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7c4d9-ad04-4826-a878-022193d6bc4c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:43.664414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:43.698869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:43.699457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.699727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.699994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.700653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:43.700947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:43.702328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:43.707119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 500/1002] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:43.712723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:43.729917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:43.749801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:43.776204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da7b79-b659-4a10-8820-0664bc5351e1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:43.813389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab712af-703d-4587-a7e5-771a5e7b39fe None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:43.815650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:43.816284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.816560 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.816821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] PATH_INFO: `/v3/services/image` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.817389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:43.818750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:43.827390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:43.828430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:43.829315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 503/1003] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Aug 12 10:38:43 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:43.834241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:43.858222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:43.859271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.859684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.860537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.861585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] RBAC: Adding query filter params (type=compute) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:43.862270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:43.864509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:43.875083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 501/1004] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Aug 12 10:38:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:43.875982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:43.938766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ef4a5-33b1-4263-b705-ba7b1d6b5757 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:43.959878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:43.979661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:43.979852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86045c-de97-426d-b3b6-f34137d7939d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:43.980288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:43.980672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:43.980943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:43.981850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] RBAC: Adding query filter params (name=image) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:43.982987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:43.984389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:43.990327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 504/1005] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:43.995746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:44.006490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:44.016443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d9dff-be7a-439b-9262-b167094ada32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:44.017159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d9dff-be7a-439b-9262-b167094ada32 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:44.017346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d9dff-be7a-439b-9262-b167094ada32 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:44.017608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d9dff-be7a-439b-9262-b167094ada32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:44.018253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d9dff-be7a-439b-9262-b167094ada32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:44.019623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d9dff-be7a-439b-9262-b167094ada32 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:44.041654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 502/1006] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:43 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:44.118432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44c0e0-af98-44cd-a84f-4b4687051f3f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:44.162176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931e81-2a59-43f2-bb89-f767db2527a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:44.162793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931e81-2a59-43f2-bb89-f767db2527a6 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:44.163068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931e81-2a59-43f2-bb89-f767db2527a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:44.163335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931e81-2a59-43f2-bb89-f767db2527a6 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:44.163964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14931e81-2a59-43f2-bb89-f767db2527a6 None admin] RBAC: Adding query filter params (type=image) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:44.164279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14931e81-2a59-43f2-bb89-f767db2527a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:44.165683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14931e81-2a59-43f2-bb89-f767db2527a6 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:44.171067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 505/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:43 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:44.449033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0660c-dde0-46ee-a336-df746b14bd74 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:44.449272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0660c-dde0-46ee-a336-df746b14bd74 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:44.449505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0660c-dde0-46ee-a336-df746b14bd74 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:44.450784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 503/1008] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:44.456558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77b542-0a7c-495f-82fc-c36a2e75ffd5 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:44.456807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77b542-0a7c-495f-82fc-c36a2e75ffd5 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:44.457098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77b542-0a7c-495f-82fc-c36a2e75ffd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:44.475271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0c77b542-0a7c-495f-82fc-c36a2e75ffd5 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:44.476496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0c77b542-0a7c-495f-82fc-c36a2e75ffd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:44.540233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:44.547597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 506/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:44 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:44.809222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0660c-dde0-46ee-a336-df746b14bd74 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:44.928741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0660c-dde0-46ee-a336-df746b14bd74 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:44.962178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:44.962787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:44.963057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:44.963318 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:44.964158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:44.965336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:44.967951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:44.975251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:44.976245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:44.981106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 504/1010] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Aug 12 10:38:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:44.984002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77b542-0a7c-495f-82fc-c36a2e75ffd5 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:45.090756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77b542-0a7c-495f-82fc-c36a2e75ffd5 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:45.126743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:45.127525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:45.127712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:45.127932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:45.128595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:45.128886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:45.130247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:45.135329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 507/1011] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:45.141532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:45.244107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26768d-c338-42d4-ada8-5f887534b1e5 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:45.281087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad0750-6860-4abd-a142-2a6de27ca039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:45.281700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0750-6860-4abd-a142-2a6de27ca039 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:45.281974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0750-6860-4abd-a142-2a6de27ca039 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:45.282241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0750-6860-4abd-a142-2a6de27ca039 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:45.282871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad0750-6860-4abd-a142-2a6de27ca039 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:45.283168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad0750-6860-4abd-a142-2a6de27ca039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:45.284592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad0750-6860-4abd-a142-2a6de27ca039 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:45.289904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 505/1012] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Aug 12 10:38:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:45.296140 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:45.359445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:45.399346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926cb76-3d2c-4c66-8ca2-88aa325c6ebb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:45.433952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903d9b1-ae25-4568-821a-ea9e342cf7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:45.434550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903d9b1-ae25-4568-821a-ea9e342cf7a4 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:45.434824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903d9b1-ae25-4568-821a-ea9e342cf7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:45.435090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903d9b1-ae25-4568-821a-ea9e342cf7a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:45.435738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903d9b1-ae25-4568-821a-ea9e342cf7a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:45.437135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903d9b1-ae25-4568-821a-ea9e342cf7a4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:45.455909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 508/1013] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:46.019662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a8ed3-d05b-4346-b3f4-2906613803ba None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.020068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a8ed3-d05b-4346-b3f4-2906613803ba None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.020171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a8ed3-d05b-4346-b3f4-2906613803ba None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.021710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 506/1014] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:46.026852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118830e-40ee-4913-b962-1da46daa4b6d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.027135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118830e-40ee-4913-b962-1da46daa4b6d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.027386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118830e-40ee-4913-b962-1da46daa4b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.045206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9118830e-40ee-4913-b962-1da46daa4b6d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:46.046372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9118830e-40ee-4913-b962-1da46daa4b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:46.062096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca419c-5f3b-4616-b99e-cf4edb970584 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.062378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca419c-5f3b-4616-b99e-cf4edb970584 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.062649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca419c-5f3b-4616-b99e-cf4edb970584 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.064110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 507/1015] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:46.068958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1776f-2752-4d48-a124-cb3bd4b10118 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.069235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1776f-2752-4d48-a124-cb3bd4b10118 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.069500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1776f-2752-4d48-a124-cb3bd4b10118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.087368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-fab1776f-2752-4d48-a124-cb3bd4b10118 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:46.088565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fab1776f-2752-4d48-a124-cb3bd4b10118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:46.128324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 509/1016] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:46 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:46.147718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:46.164305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 508/1017] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:46 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:46.389293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118830e-40ee-4913-b962-1da46daa4b6d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:46.432010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1776f-2752-4d48-a124-cb3bd4b10118 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:46.443398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:46.498053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118830e-40ee-4913-b962-1da46daa4b6d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:46.536457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:46.537106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.537380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.537643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.542747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1776f-2752-4d48-a124-cb3bd4b10118 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:46.550015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:46.555794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:46.556819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:46.558180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 510/1018] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Aug 12 10:38:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:46.563082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:46.582709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:46.583305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.583579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.583841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.584532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:46.585907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:46.604219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 509/1019] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:46 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:46.673288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe586c-5159-441c-9bac-a9b17a861ade None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:46.683708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:46.718031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:46.718634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.718905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.719168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.719820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:46.720135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:46.721557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:46.730618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 511/1020] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Aug 12 10:38:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:46.736180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:46.800212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:46.849275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184554-e2b5-4686-a55a-2458e0bde5a3 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:46.885195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:46.885843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:46.886427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:46.886693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:46.897704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:46.899217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:46.913558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:46.914551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:46.916030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:46.916030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:46.916030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:46.916030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 510/1021] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:46.920741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.028025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474da9f-b67c-41b2-a4cc-aef69790c6a7 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:47.062876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:47.065000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.065000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.065000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.065000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:47.065000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:47.066325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:47.078577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 512/1022] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:47.084398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.188610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eec76e-b860-4ca3-a4dd-e614bf4467f1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:47.195534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:47.224043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7d88-ed9a-4d54-a7f0-d2105f4212fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:47.224688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7d88-ed9a-4d54-a7f0-d2105f4212fa None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.224921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7d88-ed9a-4d54-a7f0-d2105f4212fa None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.225221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7d88-ed9a-4d54-a7f0-d2105f4212fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.233230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7d88-ed9a-4d54-a7f0-d2105f4212fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:47.234686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7d88-ed9a-4d54-a7f0-d2105f4212fa None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-308d7d88-ed9a-4d54-a7f0-d2105f4212fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:47.249202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:47.250243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:47.251214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:47.252435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 511/1023] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:38:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:47.256586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.286499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:47.345638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a453-1a02-4c71-993c-6bfaf3c0c1a4 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.345925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a453-1a02-4c71-993c-6bfaf3c0c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.346285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a453-1a02-4c71-993c-6bfaf3c0c1a4 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.347775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 512/1024] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:47.352510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d55ca3-a462-48cf-a139-345e2c7e562b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.352914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d55ca3-a462-48cf-a139-345e2c7e562b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.353020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d55ca3-a462-48cf-a139-345e2c7e562b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.369373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2ab7b-71ba-41bf-ab97-04140b525e4d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:47.371783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d5d55ca3-a462-48cf-a139-345e2c7e562b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:47.372988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d5d55ca3-a462-48cf-a139-345e2c7e562b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:47.404909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:47.405521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.405793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.406059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.406776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:47.407076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:47.408544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:47.426064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 513/1025] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:38:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:47.431700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.441280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 513/1026] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:47.516388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:47.534660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679c6f0-3a10-4bd2-beb5-9e4f16cf0a6a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:47.570695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:47.571297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.571563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.571820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.648864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:47.650323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:47.707363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d55ca3-a462-48cf-a139-345e2c7e562b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.741230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:47.760604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 514/1027] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:47 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 12 10:38:47.766643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.810276 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:47.817252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d55ca3-a462-48cf-a139-345e2c7e562b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:47.853226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:47.854143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.854617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.855164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.856396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:47.857972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:47.860283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 514/1028] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:47.865593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.871202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bd431-aeb8-49ec-b0e8-c5d94220ec83 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:47.906647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:47.907257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:47.907540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:47.907804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:47.917964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:47.919433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:47.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:47.936061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:47.937016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:47.937016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:47.937016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:47.940972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 515/1029] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:47.941980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:47.973968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8956c4-03cb-4849-b26d-c473e00adafa None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:47.977274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:48.012925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:48.013892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:48.014385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:48.014890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:48.015771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:48.017557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:48.046340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 515/1030] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:38:47 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:38:48.050615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f834-230a-44a3-80c1-2846ed6b1cf3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:48.091505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:48.092528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:48.092978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:48.093417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:48.094309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:48.094881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:48.096549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:48.111047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 516/1031] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:48.116966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:48.138109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:48.232365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6af1f4-821f-4a26-816d-57abd91d227e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:48.268456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:48.269087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:48.269392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:48.269655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:48.277257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:48.279109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:48.290568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:48.291494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:48.293455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 516/1032] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:48.297821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:48.396601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:48.403647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafaecc-1acd-4838-a4cb-03c09cd47ba8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:48.439925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b28b94-0a6f-4618-94f8-3971b218ca68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:48.440554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b28b94-0a6f-4618-94f8-3971b218ca68 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:48.440824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b28b94-0a6f-4618-94f8-3971b218ca68 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:48.441084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b28b94-0a6f-4618-94f8-3971b218ca68 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:48.441760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b28b94-0a6f-4618-94f8-3971b218ca68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:48.442084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b28b94-0a6f-4618-94f8-3971b218ca68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:48.443506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b28b94-0a6f-4618-94f8-3971b218ca68 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:48.456550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 517/1033] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:48.462427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:48.515927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467e45f-11f2-4c92-8963-964fe7fc51f8 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:48.516114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467e45f-11f2-4c92-8963-964fe7fc51f8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:48.516394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467e45f-11f2-4c92-8963-964fe7fc51f8 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:48.517868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 518/1034] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:48.522524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e870-f427-4421-a501-bc734863f513 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:48.522799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e870-f427-4421-a501-bc734863f513 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:48.523064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e870-f427-4421-a501-bc734863f513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:48.541325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0345e870-f427-4421-a501-bc734863f513 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:48.542708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0345e870-f427-4421-a501-bc734863f513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:48.553837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:48.580067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7943a-7bcf-4629-8cda-1f14d92b5e0c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:48.614999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 519/1035] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:48.615952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:48.616591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:48.616865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:48.617131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:48.617853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] RBAC: Adding query filter params (role.id=00b7ebef18e042439e2d5c6779cfae59, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a25194d34d67429b92944e0484de03cd) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:48.618156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:48.619593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:48.636623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 517/1036] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:48 2022] GET /identity/v3/role_assignments?user.id=a25194d34d67429b92944e0484de03cd&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=00b7ebef18e042439e2d5c6779cfae59 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:48.879071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345e870-f427-4421-a501-bc734863f513 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:48.967707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:48.983624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345e870-f427-4421-a501-bc734863f513 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:49.018768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:49.019447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:49.019716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:49.019980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] PATH_INFO: `/v3/services/image` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:49.020576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:49.021943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:49.030998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:49.032110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:49.037271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 520/1037] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Aug 12 10:38:48 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:49.038694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:49.143411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb8f22-2261-423e-a371-14df21f503e2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:49.177942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:49.178548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:49.178813 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:49.179070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:49.179695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] RBAC: Adding query filter params (name=image) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:49.179995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:49.181455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:49.186603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 518/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:49.192107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:49.290813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95442731-9123-4786-89d5-2c81c1f424f2 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:49.323075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:49.323784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:49.324005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:49.324240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:49.324880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] RBAC: Adding query filter params (type=image) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:49.325171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:49.326529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:49.331424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 521/1039] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:49.338839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:49.346023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:49.446855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95e894-24a1-4d42-8f09-acf79952dcdd None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:49.480227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d356eb0-68f4-42de-9971-07f48f9d0008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:49.480847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d356eb0-68f4-42de-9971-07f48f9d0008 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:49.481116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d356eb0-68f4-42de-9971-07f48f9d0008 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:49.481382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d356eb0-68f4-42de-9971-07f48f9d0008 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:49.482074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d356eb0-68f4-42de-9971-07f48f9d0008 None admin] RBAC: Adding query filter params (interface=public, service_id=3300c5c74a204d2f9960de54b81050bd, region_id=RegionOne) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:49.482373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d356eb0-68f4-42de-9971-07f48f9d0008 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:49.483740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d356eb0-68f4-42de-9971-07f48f9d0008 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:49.493369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 519/1040] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:38:49 2022] GET /identity/v3/endpoints?service_id=3300c5c74a204d2f9960de54b81050bd&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:49.494803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:49.513873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:49.604824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadf5e3-9e22-4ed7-8cb7-b99418fbe467 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:49.639378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060ab42-6dc5-4479-9f56-cb8eec14d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:49.639978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060ab42-6dc5-4479-9f56-cb8eec14d854 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:49.640245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060ab42-6dc5-4479-9f56-cb8eec14d854 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:49.640530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060ab42-6dc5-4479-9f56-cb8eec14d854 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:49.641123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060ab42-6dc5-4479-9f56-cb8eec14d854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:49.642497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060ab42-6dc5-4479-9f56-cb8eec14d854 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:49.650092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 522/1041] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:38:49 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:38:49.976435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a73c-1bf0-4c39-a082-6f1167fbe1b6 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:49.976845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a73c-1bf0-4c39-a082-6f1167fbe1b6 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:49.977030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19a73c-1bf0-4c39-a082-6f1167fbe1b6 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:49.978348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 520/1042] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:49.983802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9720267-0800-4fe8-ac67-870c5be6f64e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:49.984043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9720267-0800-4fe8-ac67-870c5be6f64e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:49.984378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9720267-0800-4fe8-ac67-870c5be6f64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:50.002967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a9720267-0800-4fe8-ac67-870c5be6f64e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:50.004153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a9720267-0800-4fe8-ac67-870c5be6f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:50.076131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 523/1043] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:50.338416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a73c-1bf0-4c39-a082-6f1167fbe1b6 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:50.411603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:50.448978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19a73c-1bf0-4c39-a082-6f1167fbe1b6 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:50.484621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:50.485239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:50.485510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:50.485776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:50.486924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:50.488407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:50.490187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 521/1044] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:50.495270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeea4c-8e64-4a41-8ebc-705ec6d7d09a None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:50.495745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:50.495822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeea4c-8e64-4a41-8ebc-705ec6d7d09a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:50.495934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeea4c-8e64-4a41-8ebc-705ec6d7d09a None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:50.497313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 524/1045] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:50.502297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8fdf-87d3-4d38-ab17-df995fa552be None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:50.502580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8fdf-87d3-4d38-ab17-df995fa552be None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:50.502856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8fdf-87d3-4d38-ab17-df995fa552be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:50.521629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-402c8fdf-87d3-4d38-ab17-df995fa552be None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:50.522847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-402c8fdf-87d3-4d38-ab17-df995fa552be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:50.524864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:50.592892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 525/1046] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:50.603823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3098e-6144-4c1c-9673-d629f417f7d8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:50.637835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:50.638435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:50.638705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:50.638968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:50.639605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:50.641089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:50.650789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:50.665646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 522/1047] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:38:50 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:38:50.853680 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c8fdf-87d3-4d38-ab17-df995fa552be None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:50.962016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c8fdf-87d3-4d38-ab17-df995fa552be None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:50.996578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:50.997208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:50.997480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:50.998225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.004944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:51.009192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:51.014932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:51.015963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:51.016870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 526/1048] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Aug 12 10:38:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:51.022532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:51.130440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1d0fd-0b6b-4f34-83ac-45b6c4a57818 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:51.165328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:51.165987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:51.166255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:51.172286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.172286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:51.172286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:51.172286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:51.183461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 523/1049] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Aug 12 10:38:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:51.184420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:51.292925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8660a9-2498-4b2f-9429-96e042db950a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:51.328279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:51.329093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:51.329457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:51.329809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.347448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:51.349037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:51.364057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:51.365160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:51.366537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:51.366537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:51.366537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:51.368609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 527/1050] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:51.370937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:51.426360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:51.475057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc2e37-b618-47c9-8476-55806383734f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:51.508360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4380b-9126-460b-b6cb-af1b1f415488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:51.508953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4380b-9126-460b-b6cb-af1b1f415488 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:51.509221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4380b-9126-460b-b6cb-af1b1f415488 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:51.509478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4380b-9126-460b-b6cb-af1b1f415488 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.510110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4380b-9126-460b-b6cb-af1b1f415488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:51.510496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4380b-9126-460b-b6cb-af1b1f415488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:51.512208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4380b-9126-460b-b6cb-af1b1f415488 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:51.523971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 524/1051] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:51.529566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:51.536674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7d4c-f07e-4556-a890-9ad905b5fb78 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:51.536958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7d4c-f07e-4556-a890-9ad905b5fb78 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:51.537227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a7d4c-f07e-4556-a890-9ad905b5fb78 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.538715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 525/1052] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:38:51.543670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1d3db-e4d1-404e-b4e9-9d66d3d9ac7a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:51.543949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1d3db-e4d1-404e-b4e9-9d66d3d9ac7a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:51.544217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1d3db-e4d1-404e-b4e9-9d66d3d9ac7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.563000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-17c1d3db-e4d1-404e-b4e9-9d66d3d9ac7a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:51.564195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-17c1d3db-e4d1-404e-b4e9-9d66d3d9ac7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:51.581901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:51.637891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 526/1053] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:38:51.640753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11772f-1f70-4ce9-9f2c-d04cc4efbc30 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:51.676020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:51.677054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:51.677528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:51.677990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.686354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:51.688001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:51.700325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:51.701295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:51.702243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:51.704705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 528/1054] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:38:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:51.710317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1d3db-e4d1-404e-b4e9-9d66d3d9ac7a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:51.758658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:51.816040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1d3db-e4d1-404e-b4e9-9d66d3d9ac7a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:51.849406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:51.850040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:51.850310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:51.850567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:51.851327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:51.851627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:51.853230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:51.872823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 527/1055] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:38:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:51.878308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:51.902460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:51.985958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798202f-b563-4976-af2a-120cf6d2fcf9 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:51.996410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:52.001404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96705bbb-1bd0-4aa4-8a98-d5212e26e8dd None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.002935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:52.022924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.023693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.024061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.024459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.035402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.036978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.039504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.040128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.040427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.040689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] PATH_INFO: `/v3/services/image` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.041260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.042671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:52.051349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:52.052320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:52.053238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:52.053238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:52.053238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:52.053424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:52.054346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 10:38:52.055230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:52.055979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 529/1056] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:52.060304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 528/1057] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Aug 12 10:38:51 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:52.060428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:52.063281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:52.166597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5de0c-fc4b-4855-89e2-32419b3cb067 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.170444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5a76-184e-470f-9205-80a28db8e74d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.201752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.202362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.202631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.203182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.203848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:52.204133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.205556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.208827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.209683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.210168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.210644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.211470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] RBAC: Adding query filter params (name=image) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:52.211964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.213553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.218008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 530/1058] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:52.223469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 529/1059] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:52.223603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:52.225021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:52.249652 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:52.257501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:52.326669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae90d-1644-4e91-b52b-c34eee6eb524 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.334939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9320fd-d6be-4901-8c26-15ef26c57ff0 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.365410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.366039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.366312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.366583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.371939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.372562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.372878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.373153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.373707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] RBAC: Adding query filter params (type=image) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:52.373805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.373999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.375242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.375348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.380909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 530/1060] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:52.386780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:52.388457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:52.389389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:52.390292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 531/1061] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:52.395089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:52.478876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:52.488965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:52.497048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63f847-fe91-4e72-9c30-72658ce71f90 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.500068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9220b5a-14da-420b-a4c8-3bc25fb6d6b1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.533448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.534085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.534272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.534538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.535219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:52.535515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.536973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.537277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc3bd0-17f5-4e95-9794-637b363317b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.538092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc3bd0-17f5-4e95-9794-637b363317b4 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.538799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc3bd0-17f5-4e95-9794-637b363317b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.539254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc3bd0-17f5-4e95-9794-637b363317b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.540570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc3bd0-17f5-4e95-9794-637b363317b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.542147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc3bd0-17f5-4e95-9794-637b363317b4 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.549824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 532/1062] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:52.555593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:52.569331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 531/1063] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:38:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:38:52.664545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb07d4-d5b0-483a-a57a-17bdb852189c None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:52.690323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88c78f-2101-4123-b4ce-ca7adcf32fe9 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.690606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88c78f-2101-4123-b4ce-ca7adcf32fe9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.690882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88c78f-2101-4123-b4ce-ca7adcf32fe9 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.692469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 532/1064] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:52.697960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17057953-8251-4000-ac5c-b6c85c455ffa None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.698243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17057953-8251-4000-ac5c-b6c85c455ffa None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.698513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17057953-8251-4000-ac5c-b6c85c455ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.712176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:52.712879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:52.713151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:52.713468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a25194d34d67429b92944e0484de03cd/roles/00b7ebef18e042439e2d5c6779cfae59` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:52.715633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=a25194d34d67429b92944e0484de03cd, role_id=00b7ebef18e042439e2d5c6779cfae59)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:52.717105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:52.718241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-17057953-8251-4000-ac5c-b6c85c455ffa None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:52.719437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-17057953-8251-4000-ac5c-b6c85c455ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:52.726338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:52.733365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 533/1065] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:38:52 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/a25194d34d67429b92944e0484de03cd/roles/00b7ebef18e042439e2d5c6779cfae59 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:38:52.803890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 533/1066] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:53.065386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:53.170478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492d59f-f345-4e4c-8d93-1b8669a9c212 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:53.203428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:53.204055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:53.204351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:53.204622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:53.215495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:53.220090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:53.221180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:53.222037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:53.224591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 534/1067] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:53.227248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17057953-8251-4000-ac5c-b6c85c455ffa None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:53.309185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:53.331207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17057953-8251-4000-ac5c-b6c85c455ffa None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:53.370952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:53.371554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:53.371828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:53.372094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:53.372815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:53.373116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:53.374494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:53.387700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 534/1068] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:53.388870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:53.449437 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:53.493890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb371719-9766-4b8e-90e2-d1e50e7f5168 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:53.527896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:53.528609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:53.528882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:53.529149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] PATH_INFO: `/v3/users/cb8ab1609fc04685badd1e3a44b78165` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:53.542990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] RBAC: Authorizing `identity:get_user(user_id=cb8ab1609fc04685badd1e3a44b78165)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:53.544544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:53.551212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 535/1069] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:38:53 2022] GET /identity/v3/users/cb8ab1609fc04685badd1e3a44b78165 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:53.552597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:53.658293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547068c-db7d-477b-982d-598f10a7a289 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:53.694619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:53.695237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:53.695506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:53.695769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:53.709390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:53.768978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:53.770660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:53.873008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 535/1070] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:53.878657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:53.975069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41020-cb32-44c7-b4a0-3f72b33c2766 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:53.991316 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:54.010356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0846f2-2ba6-4406-806b-5b3bff5c2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:54.011297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0846f2-2ba6-4406-806b-5b3bff5c2fbd None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:54.011560 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0846f2-2ba6-4406-806b-5b3bff5c2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:54.011822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0846f2-2ba6-4406-806b-5b3bff5c2fbd None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:54.012938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0846f2-2ba6-4406-806b-5b3bff5c2fbd None admin] RBAC: Authorizing `identity:get_project(project_id=6e9bcaa4a34b4ebfa117d278df6496ee)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:54.014376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0846f2-2ba6-4406-806b-5b3bff5c2fbd None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:54.021477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 536/1071] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:38:53 2022] GET /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:54.022869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:54.065943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:54.124234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d273d-37e5-4a7c-9aef-f5f5a31caded None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:54.158994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ada835-d102-4a95-8b91-47fd57a36775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:54.159596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada835-d102-4a95-8b91-47fd57a36775 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:54.159870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada835-d102-4a95-8b91-47fd57a36775 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:54.160140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada835-d102-4a95-8b91-47fd57a36775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:54.160896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada835-d102-4a95-8b91-47fd57a36775 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee, user.id=cb8ab1609fc04685badd1e3a44b78165) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:54.161203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada835-d102-4a95-8b91-47fd57a36775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:54.162640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada835-d102-4a95-8b91-47fd57a36775 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:54.168529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 536/1072] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:54 2022] GET /identity/v3/role_assignments?user.id=cb8ab1609fc04685badd1e3a44b78165&scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:54.517798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146346c-7d2b-46ef-954b-06182af82a74 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:54.519960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146346c-7d2b-46ef-954b-06182af82a74 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:54.519960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146346c-7d2b-46ef-954b-06182af82a74 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:54.519960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 537/1073] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:54.525480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae213-a446-40ef-9d9f-c5a6b1c4c105 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:54.525744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae213-a446-40ef-9d9f-c5a6b1c4c105 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:54.526070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae213-a446-40ef-9d9f-c5a6b1c4c105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:54.545223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f24ae213-a446-40ef-9d9f-c5a6b1c4c105 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:54.546392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f24ae213-a446-40ef-9d9f-c5a6b1c4c105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:54.615902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 537/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:54.623409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc88f7-8099-4707-a538-9745de99142b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:54.623876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc88f7-8099-4707-a538-9745de99142b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:54.623961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc88f7-8099-4707-a538-9745de99142b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:54.625447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 538/1075] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:54.630575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5204597-27f3-441d-9e45-9d3e97186146 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:54.630832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5204597-27f3-441d-9e45-9d3e97186146 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:54.631133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5204597-27f3-441d-9e45-9d3e97186146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:54.649747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b5204597-27f3-441d-9e45-9d3e97186146 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:54.651040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b5204597-27f3-441d-9e45-9d3e97186146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:54.690074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:54.729959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 538/1076] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:54.879320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc88f7-8099-4707-a538-9745de99142b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:54.981913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc88f7-8099-4707-a538-9745de99142b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:54.995534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5204597-27f3-441d-9e45-9d3e97186146 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.020493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.021188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.021452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.021720 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.076917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:55.094835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:55.096347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:55.102916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5204597-27f3-441d-9e45-9d3e97186146 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:55.139728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.140303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.140577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.140845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.151767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:55.156556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:55.157579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:55.158543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 539/1077] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Aug 12 10:38:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:55.163360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.191603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:55.205446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 539/1078] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:55.211918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.273034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57028c33-82c3-42c7-93b2-eeb0f61f5a37 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:55.304391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:55.308955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.309227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.309522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.309913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.310589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:55.310888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:55.312364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:55.321201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 540/1079] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Aug 12 10:38:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:55.321977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32362849-62fe-4cf8-b0dd-5e8cc6046805 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:55.326974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.362180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.362828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.363082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.363348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.374073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:55.375880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:55.391574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:55.392728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:55.394078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:55.394078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:55.394078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:55.394078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 540/1080] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:55.398555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.425300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:55.439011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5c435-ae35-4e35-962c-adb9347599fc None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:55.475329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.475951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.476224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.476517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.488523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:55.489973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:55.500027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bcd90-4ffd-4679-9847-4623d996c2dd None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:55.505210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:55.506170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:55.507446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:55.507446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:55.507446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:55.507446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 541/1081] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:55.512553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.547942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.549101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.549541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.550733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.551572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:55.552025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:55.553772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:55.557867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:55.572015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 541/1082] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:55.634980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac4232-7949-4921-9d3e-25ec20a3af80 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:55.674064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.674677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.674945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.675206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.675855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:55.676146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:55.677572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:55.688046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:55.692831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 542/1083] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:55.698646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.805153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bfe8d-0268-40b9-afc3-b8814cb93325 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:55.840740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:55.841367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:55.841637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:55.841898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:55.850360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:55.851790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:38:55.863242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:55.864469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:55.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 542/1084] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Aug 12 10:38:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:55.870264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:55.970521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ceed0-0541-406d-8cc8-8ce6af223c63 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.010608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b1010-6169-418e-9d5e-5c03a1fe2ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.011533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b1010-6169-418e-9d5e-5c03a1fe2ab2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.011793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b1010-6169-418e-9d5e-5c03a1fe2ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.012054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b1010-6169-418e-9d5e-5c03a1fe2ab2 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.012803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b1010-6169-418e-9d5e-5c03a1fe2ab2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:56.013099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b1010-6169-418e-9d5e-5c03a1fe2ab2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:56.014511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b1010-6169-418e-9d5e-5c03a1fe2ab2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:56.032622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 543/1085] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Aug 12 10:38:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:56.038456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.044174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be374-4fdb-4840-a2c6-07308f33bed7 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.044492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be374-4fdb-4840-a2c6-07308f33bed7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.044763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97be374-4fdb-4840-a2c6-07308f33bed7 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.046083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 544/1086] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:56.051104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ac31-8670-4910-8dc5-0e381fe5efab None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.051384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ac31-8670-4910-8dc5-0e381fe5efab None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.051650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ac31-8670-4910-8dc5-0e381fe5efab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.070097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-5788ac31-8670-4910-8dc5-0e381fe5efab None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:56.071289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5788ac31-8670-4910-8dc5-0e381fe5efab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:56.098582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:56.124730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:56.139305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 545/1087] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:56.145329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87172ba5-6dda-447a-87e5-b242c42c61d0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.178607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.179202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.179473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.179738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.246923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:56.248299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:56.305541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:56.362005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 543/1088] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:38:56.363284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788ac31-8670-4910-8dc5-0e381fe5efab None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.399665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.464841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788ac31-8670-4910-8dc5-0e381fe5efab None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.500289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.500831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.501105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.501268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787ef4a-779a-4973-9660-eb5a31f227c8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.501364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.511922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:56.513412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:38:56.528171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:56.529422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:38:56.530333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:38:56.530333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:38:56.530333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:56.530333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 546/1089] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:56.536576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.543007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.543633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.543907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.544172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.554830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:56.559634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:56.560651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:38:56.561712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 544/1090] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:38:56.566660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.579043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:56.585284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:56.640619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240cea6-e695-40bd-b63d-8c8d7beb3716 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.668908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd342d-a941-4183-83a4-cd169f12769e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.673966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.674569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.674844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.675113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.675764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:56.676063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:56.677464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:56.689262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 547/1091] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:56.695006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.702581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.703181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.703454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.703718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.704400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:56.704696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:56.706085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:56.714865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 545/1092] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:56.720593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.790996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:56.793998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e09f8-8fbd-4590-8356-592a307ed97d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.798242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:56.821283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499da3e7-8cf4-4aa6-96f0-2b2eb19081e0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.829129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.829782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.830056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.830293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.836875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:56.838440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:38:56.849663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:56.850590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:56.851508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 548/1093] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:56.857145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.858031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.858677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.858953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.859216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] PATH_INFO: `/v3/users/cb8ab1609fc04685badd1e3a44b78165` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.860380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] RBAC: Authorizing `identity:get_user(user_id=cb8ab1609fc04685badd1e3a44b78165)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:56.861795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:56.863360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 546/1094] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/users/cb8ab1609fc04685badd1e3a44b78165 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:56.868953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:56.955111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb498b-7f5b-495d-aad0-eab03e3dd3e9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.974169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c42de-8eb6-432b-9b8e-835449120589 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:56.997037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:56.997632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:56.997902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:56.998161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:56.998843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:56.999136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:57.000587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:57.012061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 549/1095] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:57.014621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:57.015449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:57.015722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:57.016094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:57.017371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] RBAC: Authorizing `identity:get_project(project_id=6e9bcaa4a34b4ebfa117d278df6496ee)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:57.018855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:57.018965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:57.020592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 547/1096] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:38:56 2022] GET /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:38:57.026190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:57.042117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:57.042868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:57.120961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6f17b-0435-4fdf-9dbe-5c1cb345b0c9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:57.131024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48568100-3508-4f30-8d7b-6ff315f6bb32 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:57.154335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299c6aa-1875-4b2e-a403-4197bec51255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:57.154947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299c6aa-1875-4b2e-a403-4197bec51255 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:57.155214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299c6aa-1875-4b2e-a403-4197bec51255 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:57.155474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299c6aa-1875-4b2e-a403-4197bec51255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:57.156188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299c6aa-1875-4b2e-a403-4197bec51255 None admin] RBAC: Adding query filter params (role.id=00b7ebef18e042439e2d5c6779cfae59, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=a25194d34d67429b92944e0484de03cd) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:57.156514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299c6aa-1875-4b2e-a403-4197bec51255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:57.157948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299c6aa-1875-4b2e-a403-4197bec51255 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:57.163500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 550/1097] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:38:57 2022] GET /identity/v3/role_assignments?user.id=a25194d34d67429b92944e0484de03cd&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=00b7ebef18e042439e2d5c6779cfae59 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:57.165068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ceee8-e25f-4d48-94e8-9175376b296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:57.165720 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ceee8-e25f-4d48-94e8-9175376b296f None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:57.165996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ceee8-e25f-4d48-94e8-9175376b296f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:57.166265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ceee8-e25f-4d48-94e8-9175376b296f None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee/users/cb8ab1609fc04685badd1e3a44b78165/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:57.168222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ceee8-e25f-4d48-94e8-9175376b296f None admin] RBAC: Authorizing `identity:create_grant(project_id=6e9bcaa4a34b4ebfa117d278df6496ee, user_id=cb8ab1609fc04685badd1e3a44b78165, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:57.169688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ceee8-e25f-4d48-94e8-9175376b296f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:57.185255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 548/1098] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:38:57 2022] PUT /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee/users/cb8ab1609fc04685badd1e3a44b78165/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:38:59.059801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fb9ab-1481-4c5f-b552-99d6cecc348e None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:59.060417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fb9ab-1481-4c5f-b552-99d6cecc348e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:59.060417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fb9ab-1481-4c5f-b552-99d6cecc348e None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:59.061727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 551/1099] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:38:59.067991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c94c0-0c64-4879-8d79-9a73bcef1e22 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:59.068658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c94c0-0c64-4879-8d79-9a73bcef1e22 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:59.069071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c94c0-0c64-4879-8d79-9a73bcef1e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:59.095066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-072c94c0-0c64-4879-8d79-9a73bcef1e22 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:38:59.096236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-072c94c0-0c64-4879-8d79-9a73bcef1e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:38:59.168279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 549/1100] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:38:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:38:59.434641 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fb9ab-1481-4c5f-b552-99d6cecc348e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:59.522978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:59.540194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fb9ab-1481-4c5f-b552-99d6cecc348e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:59.574127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:59.574726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:59.574992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:59.575251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:59.586361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:38:59.591058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:38:59.592124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:38:59.593483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 552/1101] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:38:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:38:59.601905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c94c0-0c64-4879-8d79-9a73bcef1e22 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:59.606279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:38:59.709278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c94c0-0c64-4879-8d79-9a73bcef1e22 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:59.744369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:59.745029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:59.745294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:59.745554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:59.746200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:38:59.746496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:59.747854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:59.755990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 550/1102] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:38:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:59.763417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:59.884614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566400-85df-447c-af55-6c8f876a0457 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:38:59.919799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:38:59.920432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:38:59.920705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:38:59.920976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] PATH_INFO: `/v3/users/cb8ab1609fc04685badd1e3a44b78165` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:38:59.922217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] RBAC: Authorizing `identity:get_user(user_id=cb8ab1609fc04685badd1e3a44b78165)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:38:59.923666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:38:59.925269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 553/1103] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:38:59 2022] GET /identity/v3/users/cb8ab1609fc04685badd1e3a44b78165 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:38:59.931361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:38:59.995505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:00.031782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978ad0e-96f6-4f16-a3fa-d77570d35f71 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:00.067888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:00.068505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:00.069142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:00.069142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:00.141462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:00.142833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:00.211757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:00.261180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 551/1104] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:38:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:39:00.267066 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:00.284077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:00.384038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ef4d1-b639-4639-83d8-be84b2f94a02 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:00.420283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de5393-f4f4-4d3d-915a-68d552dfe47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:00.420908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5393-f4f4-4d3d-915a-68d552dfe47a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:00.421180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5393-f4f4-4d3d-915a-68d552dfe47a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:00.421447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5393-f4f4-4d3d-915a-68d552dfe47a None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:00.422601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de5393-f4f4-4d3d-915a-68d552dfe47a None admin] RBAC: Authorizing `identity:get_project(project_id=6e9bcaa4a34b4ebfa117d278df6496ee)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:00.424061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de5393-f4f4-4d3d-915a-68d552dfe47a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:00.425713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 554/1105] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:00 2022] GET /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:00.435776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:00.543912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808a289-a037-48e2-b8b5-da95402769b0 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:00.578548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1da43-b57c-4fc3-beec-360d42dd498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:00.579124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1da43-b57c-4fc3-beec-360d42dd498f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:00.579390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1da43-b57c-4fc3-beec-360d42dd498f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:00.579653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1da43-b57c-4fc3-beec-360d42dd498f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:00.580382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1da43-b57c-4fc3-beec-360d42dd498f None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee, user.id=cb8ab1609fc04685badd1e3a44b78165) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:00.580677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1da43-b57c-4fc3-beec-360d42dd498f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:00.582090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1da43-b57c-4fc3-beec-360d42dd498f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:00.587873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 552/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:39:00 2022] GET /identity/v3/role_assignments?user.id=cb8ab1609fc04685badd1e3a44b78165&scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:02.485015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208a973-b798-483a-9208-7dbbf1c081df None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:02.485890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208a973-b798-483a-9208-7dbbf1c081df None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:02.485890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208a973-b798-483a-9208-7dbbf1c081df None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:02.487992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 555/1107] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:02.496076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a5c1c-e925-416b-8766-b00dd4beb717 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:02.496545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a5c1c-e925-416b-8766-b00dd4beb717 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:02.496633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a5c1c-e925-416b-8766-b00dd4beb717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:02.516241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2c0a5c1c-e925-416b-8766-b00dd4beb717 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:02.517424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2c0a5c1c-e925-416b-8766-b00dd4beb717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:02.541530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:02.586030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 553/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:39:02.851146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208a973-b798-483a-9208-7dbbf1c081df None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:02.947188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:02.969542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208a973-b798-483a-9208-7dbbf1c081df None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:03.002542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c3bc5-865c-4adc-9c25-9ecd3e90ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:03.003148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3bc5-865c-4adc-9c25-9ecd3e90ce57 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:03.003419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3bc5-865c-4adc-9c25-9ecd3e90ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:03.003685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c3bc5-865c-4adc-9c25-9ecd3e90ce57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:03.004920 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c3bc5-865c-4adc-9c25-9ecd3e90ce57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:03.006463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c3bc5-865c-4adc-9c25-9ecd3e90ce57 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:03.008014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 556/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:39:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:03.015006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a5c1c-e925-416b-8766-b00dd4beb717 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:03.130990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a5c1c-e925-416b-8766-b00dd4beb717 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:03.169048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e761e-7986-492c-b0c0-e7390c36d81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:03.169638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e761e-7986-492c-b0c0-e7390c36d81d None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:03.169888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e761e-7986-492c-b0c0-e7390c36d81d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:03.170182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e761e-7986-492c-b0c0-e7390c36d81d None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:03.170813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e761e-7986-492c-b0c0-e7390c36d81d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:03.172249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e761e-7986-492c-b0c0-e7390c36d81d None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:03.200804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 554/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:39:03 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:39:05.150966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ebad4-fcd2-489a-b103-bb06d78a21bc None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:05.151561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ebad4-fcd2-489a-b103-bb06d78a21bc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:05.151561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ebad4-fcd2-489a-b103-bb06d78a21bc None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:05.153030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 557/1111] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:05.159394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ba942-c456-49cf-9395-7b2a209cbb49 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:05.160104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ba942-c456-49cf-9395-7b2a209cbb49 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:05.160303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ba942-c456-49cf-9395-7b2a209cbb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:05.188609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-549ba942-c456-49cf-9395-7b2a209cbb49 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:05.189798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-549ba942-c456-49cf-9395-7b2a209cbb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:05.200867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:05.259112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 555/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:05.538295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ebad4-fcd2-489a-b103-bb06d78a21bc None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:05.648158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ebad4-fcd2-489a-b103-bb06d78a21bc None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:05.688049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:05.688738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:05.689011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:05.689278 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] PATH_INFO: `/v3/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:05.690796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] RBAC: Authorizing `identity:get_role(role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:05.692167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:05.693519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 558/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:05 2022] GET /identity/v3/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:05.700732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ba942-c456-49cf-9395-7b2a209cbb49 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:05.814633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ba942-c456-49cf-9395-7b2a209cbb49 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:05.854421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:05.855035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:05.855311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:05.855580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] PATH_INFO: `/v3/users/a2dfb66523d54022a0c2350dc34a7f9c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:05.868712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] RBAC: Authorizing `identity:get_user(user_id=a2dfb66523d54022a0c2350dc34a7f9c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:05.870259 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:05.871826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 556/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:05 2022] GET /identity/v3/users/a2dfb66523d54022a0c2350dc34a7f9c => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:05.881118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:05.908060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:06.001064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e30b1-f75c-4880-bf3a-015197a72580 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:06.036114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:06.036730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:06.037000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:06.037265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:06.124127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:06.125545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:06.143653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:06.227934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 559/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:39:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:39:06.235011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:06.239506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:06.338540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76b19-d12f-4078-9517-c03bc10b9088 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:06.375853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987e48f-11bc-4d4a-a526-82aaf6956c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:06.376461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987e48f-11bc-4d4a-a526-82aaf6956c44 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:06.376736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987e48f-11bc-4d4a-a526-82aaf6956c44 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:06.377004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987e48f-11bc-4d4a-a526-82aaf6956c44 None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:06.378157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987e48f-11bc-4d4a-a526-82aaf6956c44 None admin] RBAC: Authorizing `identity:get_project(project_id=6e9bcaa4a34b4ebfa117d278df6496ee)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:06.379601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987e48f-11bc-4d4a-a526-82aaf6956c44 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:06.381261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 557/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:06 2022] GET /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:06.389094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:06.504743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be6fd-30b6-477d-bea8-ba9a7b65f18f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:06.521775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:06.543061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2a08-7448-46eb-9473-eb9736d639fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:06.543649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2a08-7448-46eb-9473-eb9736d639fe None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:06.543923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2a08-7448-46eb-9473-eb9736d639fe None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:06.544189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2a08-7448-46eb-9473-eb9736d639fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:06.544945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2a08-7448-46eb-9473-eb9736d639fe None admin] RBAC: Adding query filter params (role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee, user.id=a2dfb66523d54022a0c2350dc34a7f9c) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:06.545243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2a08-7448-46eb-9473-eb9736d639fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:06.546670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2a08-7448-46eb-9473-eb9736d639fe None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:06.552029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 560/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:39:06 2022] GET /identity/v3/role_assignments?user.id=a2dfb66523d54022a0c2350dc34a7f9c&scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:08.464055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a9bf-eb7b-4127-ac75-6ca6bca6545c None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:08.464691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a9bf-eb7b-4127-ac75-6ca6bca6545c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:08.464967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a9bf-eb7b-4127-ac75-6ca6bca6545c None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:08.466588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 558/1118] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:08.474168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200bddf-be18-43f1-b900-6b7306dd5fec None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:08.474659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200bddf-be18-43f1-b900-6b7306dd5fec None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:08.474659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200bddf-be18-43f1-b900-6b7306dd5fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:08.499198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6200bddf-be18-43f1-b900-6b7306dd5fec None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:08.501028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6200bddf-be18-43f1-b900-6b7306dd5fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:08.585631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 561/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:08.861447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38a9bf-eb7b-4127-ac75-6ca6bca6545c None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:08.947747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:08.986467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38a9bf-eb7b-4127-ac75-6ca6bca6545c None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:09.023551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:09.024164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:09.024468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:09.024729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] PATH_INFO: `/v3/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:09.026283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] RBAC: Authorizing `identity:get_role(role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:09.027722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:09.029077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 559/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:08 2022] GET /identity/v3/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:09.037906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200bddf-be18-43f1-b900-6b7306dd5fec None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:09.141850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200bddf-be18-43f1-b900-6b7306dd5fec None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:09.152860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:09.178659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:09.179265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:09.179528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:09.179791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] PATH_INFO: `/v3/users/a2dfb66523d54022a0c2350dc34a7f9c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:09.181552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] RBAC: Authorizing `identity:get_user(user_id=a2dfb66523d54022a0c2350dc34a7f9c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:09.183248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:09.184859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 562/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:09 2022] GET /identity/v3/users/a2dfb66523d54022a0c2350dc34a7f9c => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:09.191123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:09.305751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d416-4985-4e9b-840c-4aaa7114212b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:09.344175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8f86b-2be5-44cd-9170-d046390904ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:09.344824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f86b-2be5-44cd-9170-d046390904ce None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:09.345094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f86b-2be5-44cd-9170-d046390904ce None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:09.345360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f86b-2be5-44cd-9170-d046390904ce None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:09.346485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8f86b-2be5-44cd-9170-d046390904ce None admin] RBAC: Authorizing `identity:get_project(project_id=6e9bcaa4a34b4ebfa117d278df6496ee)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:09.347931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8f86b-2be5-44cd-9170-d046390904ce None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:09.349615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 560/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:09 2022] GET /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:09.356019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:09.457229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e159a9-d573-4028-af70-8bbbcad51710 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:09.492411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d12942-00cb-437e-96bc-d3178d8bf356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:09.493010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d12942-00cb-437e-96bc-d3178d8bf356 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:09.493304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d12942-00cb-437e-96bc-d3178d8bf356 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:09.493564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d12942-00cb-437e-96bc-d3178d8bf356 None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee/users/a2dfb66523d54022a0c2350dc34a7f9c/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:09.496311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d12942-00cb-437e-96bc-d3178d8bf356 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e9bcaa4a34b4ebfa117d278df6496ee, user_id=a2dfb66523d54022a0c2350dc34a7f9c, role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:09.497758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d12942-00cb-437e-96bc-d3178d8bf356 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:09.514808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 563/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:39:09 2022] PUT /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee/users/a2dfb66523d54022a0c2350dc34a7f9c/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:39:11.393555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2d22d-537b-44ec-87d1-3b70a83c9d02 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:11.394107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2d22d-537b-44ec-87d1-3b70a83c9d02 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:11.394107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2d22d-537b-44ec-87d1-3b70a83c9d02 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:11.395405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 561/1124] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:11.402079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409787de-ea6e-4f1f-9d3a-3114c67e4f17 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:11.402761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409787de-ea6e-4f1f-9d3a-3114c67e4f17 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:11.402967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409787de-ea6e-4f1f-9d3a-3114c67e4f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:11.433343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-409787de-ea6e-4f1f-9d3a-3114c67e4f17 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:11.434511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-409787de-ea6e-4f1f-9d3a-3114c67e4f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:11.464684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:11.507349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 564/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:11.783405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2d22d-537b-44ec-87d1-3b70a83c9d02 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:11.797213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:11.891998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2d22d-537b-44ec-87d1-3b70a83c9d02 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:11.926531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:11.927131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:11.927398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:11.927661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] PATH_INFO: `/v3/roles/81fa4ea7f9144b4f917a47cfbd076d4f` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:11.929153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] RBAC: Authorizing `identity:get_role(role_id=81fa4ea7f9144b4f917a47cfbd076d4f)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:11.930505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:11.931918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 562/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:11 2022] GET /identity/v3/roles/81fa4ea7f9144b4f917a47cfbd076d4f => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:11.938822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409787de-ea6e-4f1f-9d3a-3114c67e4f17 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:12.055993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409787de-ea6e-4f1f-9d3a-3114c67e4f17 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:12.091938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:12.092571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:12.092839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:12.093110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] PATH_INFO: `/v3/users/a2dfb66523d54022a0c2350dc34a7f9c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:12.094294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] RBAC: Authorizing `identity:get_user(user_id=a2dfb66523d54022a0c2350dc34a7f9c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:12.095728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:12.097311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 565/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:11 2022] GET /identity/v3/users/a2dfb66523d54022a0c2350dc34a7f9c => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:12.107524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:12.206315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:12.236979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ea0ac-b843-4b28-839e-214aa7aedd66 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:12.279584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:12.280513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:12.280862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:12.281213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:12.357209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:12.358676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:12.436203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:12.464921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 563/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:39:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:39:12.471445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:12.506875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:12.590478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ad3ca-3dd4-42b8-ac5d-c58921868b5c None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:12.626373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b3413-dfe1-4c97-9dde-667d7a7bcd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:12.626972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b3413-dfe1-4c97-9dde-667d7a7bcd59 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:12.627238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b3413-dfe1-4c97-9dde-667d7a7bcd59 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:12.627505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b3413-dfe1-4c97-9dde-667d7a7bcd59 None admin] PATH_INFO: `/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:12.628749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b3413-dfe1-4c97-9dde-667d7a7bcd59 None admin] RBAC: Authorizing `identity:get_project(project_id=6e9bcaa4a34b4ebfa117d278df6496ee)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:12.630173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b3413-dfe1-4c97-9dde-667d7a7bcd59 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:12.631769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 566/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:12 2022] GET /identity/v3/projects/6e9bcaa4a34b4ebfa117d278df6496ee => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:12.639239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:12.747255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5388076-fe28-41db-9774-7309b0cf1986 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:12.783343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d4d3a-c60d-4546-8d94-3ad1065b3a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:12.783933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d4d3a-c60d-4546-8d94-3ad1065b3a4f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:12.784202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d4d3a-c60d-4546-8d94-3ad1065b3a4f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:12.784493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d4d3a-c60d-4546-8d94-3ad1065b3a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:12.785201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d4d3a-c60d-4546-8d94-3ad1065b3a4f None admin] RBAC: Adding query filter params (role.id=81fa4ea7f9144b4f917a47cfbd076d4f, scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee, user.id=a2dfb66523d54022a0c2350dc34a7f9c) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:12.785507 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d4d3a-c60d-4546-8d94-3ad1065b3a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:12.786914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d4d3a-c60d-4546-8d94-3ad1065b3a4f None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:12.792839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 564/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:39:12 2022] GET /identity/v3/role_assignments?user.id=a2dfb66523d54022a0c2350dc34a7f9c&scope.project.id=6e9bcaa4a34b4ebfa117d278df6496ee&role.id=81fa4ea7f9144b4f917a47cfbd076d4f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:14.723486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c33ea-16fc-4537-90ac-66bbe8fe655c None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:14.724353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c33ea-16fc-4537-90ac-66bbe8fe655c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:14.724353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c33ea-16fc-4537-90ac-66bbe8fe655c None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:14.726814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 567/1131] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:14.732289 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdcd24-1079-4f79-8ace-b174683cc9cb None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:14.732749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdcd24-1079-4f79-8ace-b174683cc9cb None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:14.732847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdcd24-1079-4f79-8ace-b174683cc9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:14.752421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-54bdcd24-1079-4f79-8ace-b174683cc9cb None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:14.753637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-54bdcd24-1079-4f79-8ace-b174683cc9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:14.792651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:14.825852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 565/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:15.087323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c33ea-16fc-4537-90ac-66bbe8fe655c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:15.207966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c33ea-16fc-4537-90ac-66bbe8fe655c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:15.210281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:15.244778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbdf2a-6a07-492a-94c4-2a0f56390a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:15.245385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbdf2a-6a07-492a-94c4-2a0f56390a47 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:15.245650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbdf2a-6a07-492a-94c4-2a0f56390a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:15.245913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbdf2a-6a07-492a-94c4-2a0f56390a47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:15.247086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbdf2a-6a07-492a-94c4-2a0f56390a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:15.248536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbdf2a-6a07-492a-94c4-2a0f56390a47 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:15.250099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 568/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:39:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:15.256054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdcd24-1079-4f79-8ace-b174683cc9cb None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:15.361536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdcd24-1079-4f79-8ace-b174683cc9cb None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:15.396475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279c83f-86ea-4454-be8e-5a31e1c7bdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:15.397061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279c83f-86ea-4454-be8e-5a31e1c7bdba None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:15.397328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279c83f-86ea-4454-be8e-5a31e1c7bdba None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:15.397593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279c83f-86ea-4454-be8e-5a31e1c7bdba None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:15.398215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279c83f-86ea-4454-be8e-5a31e1c7bdba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:15.399634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279c83f-86ea-4454-be8e-5a31e1c7bdba None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:15.434265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 566/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:39:15 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:39:17.346099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac12075-fd6c-4330-926b-89bcd13d51dd None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:17.346567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac12075-fd6c-4330-926b-89bcd13d51dd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:17.346657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac12075-fd6c-4330-926b-89bcd13d51dd None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:17.348806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 569/1135] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:17.357216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78369a0-a2e1-4d63-a235-a664eace07c3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:17.357642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78369a0-a2e1-4d63-a235-a664eace07c3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:17.357757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78369a0-a2e1-4d63-a235-a664eace07c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:17.376680 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a78369a0-a2e1-4d63-a235-a664eace07c3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:17.377856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a78369a0-a2e1-4d63-a235-a664eace07c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:17.394646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:17.446090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 567/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:39:17.725878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac12075-fd6c-4330-926b-89bcd13d51dd None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:17.833229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac12075-fd6c-4330-926b-89bcd13d51dd None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:17.877361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f2387-1f49-4976-a8fa-cde222757a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:17.878232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2387-1f49-4976-a8fa-cde222757a02 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:17.878571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2387-1f49-4976-a8fa-cde222757a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:17.878901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2387-1f49-4976-a8fa-cde222757a02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:17.880887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2387-1f49-4976-a8fa-cde222757a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:17.883153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2387-1f49-4976-a8fa-cde222757a02 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:17.886276 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 570/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:17.892419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78369a0-a2e1-4d63-a235-a664eace07c3 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:17.997156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78369a0-a2e1-4d63-a235-a664eace07c3 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:18.032248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b040b-d27e-45db-83cc-f7e0a0662057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:18.032859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b040b-d27e-45db-83cc-f7e0a0662057 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:18.033129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b040b-d27e-45db-83cc-f7e0a0662057 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:18.033393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b040b-d27e-45db-83cc-f7e0a0662057 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:18.034023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b040b-d27e-45db-83cc-f7e0a0662057 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:18.035463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b040b-d27e-45db-83cc-f7e0a0662057 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:18.057727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 568/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:39:17 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:39:19.983718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de2ca0-74ce-4f11-9dd1-9039af541d2a None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:19.984774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de2ca0-74ce-4f11-9dd1-9039af541d2a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:19.984774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de2ca0-74ce-4f11-9dd1-9039af541d2a None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:19.986765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 571/1139] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:19.996109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22b0d-75c6-45c4-984a-8698a65f7a0d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:19.996543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22b0d-75c6-45c4-984a-8698a65f7a0d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:19.996750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea22b0d-75c6-45c4-984a-8698a65f7a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:20.021184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7ea22b0d-75c6-45c4-984a-8698a65f7a0d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:20.022459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7ea22b0d-75c6-45c4-984a-8698a65f7a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:20.024420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:20.094105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 569/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:20.365656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de2ca0-74ce-4f11-9dd1-9039af541d2a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:20.423359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:20.485735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de2ca0-74ce-4f11-9dd1-9039af541d2a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:20.525237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:20.525847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:20.526122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:20.526367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:20.537920 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:39:20.542679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:39:20.543868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:39:20.544816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 572/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:39:20.552131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22b0d-75c6-45c4-984a-8698a65f7a0d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:20.660024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea22b0d-75c6-45c4-984a-8698a65f7a0d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:20.697857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:20.698454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:20.698727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:20.698992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:20.699660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:20.699935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:20.701344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:20.706446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:20.709922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 570/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:20.715847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:20.817947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:20.821068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df61b00-3076-4289-9e90-3d284b2ea6bb None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:20.857947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:20.858553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:20.858826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:20.859095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] PATH_INFO: `/v3/users/dfc790aa3d3f4b8ead34a882e6d88224` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:20.873025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] RBAC: Authorizing `identity:get_user(user_id=dfc790aa3d3f4b8ead34a882e6d88224)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:20.874466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:20.876041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 573/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:20 2022] GET /identity/v3/users/dfc790aa3d3f4b8ead34a882e6d88224 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:20.882352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:20.983368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630a51-aafb-4946-9441-f6a4ee178744 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:21.018838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:21.019670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:21.019994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:21.020389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:21.095690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:21.098023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:21.099859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:21.201961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 571/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:39:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 12 10:39:21.210816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:21.320832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646c3f7-7e5b-4739-bc6b-3d422cd6ad62 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:21.357639 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b843365-a5ce-48d0-abc3-34e94cf002fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:21.358266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b843365-a5ce-48d0-abc3-34e94cf002fa None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:21.358541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b843365-a5ce-48d0-abc3-34e94cf002fa None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:21.358807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b843365-a5ce-48d0-abc3-34e94cf002fa None admin] PATH_INFO: `/v3/projects/9b6af8e4ca3048539f554280c06818af` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:21.360007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b843365-a5ce-48d0-abc3-34e94cf002fa None admin] RBAC: Authorizing `identity:get_project(project_id=9b6af8e4ca3048539f554280c06818af)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:21.361513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b843365-a5ce-48d0-abc3-34e94cf002fa None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:21.363156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 574/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:21 2022] GET /identity/v3/projects/9b6af8e4ca3048539f554280c06818af => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:21.369502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:21.469000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:21.472203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bcf8d-beb5-4f87-8fef-4f70f454141e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:21.506840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd70d1-eaad-448f-8b79-d23423ae8f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:21.507433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd70d1-eaad-448f-8b79-d23423ae8f06 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:21.507708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd70d1-eaad-448f-8b79-d23423ae8f06 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:21.507971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd70d1-eaad-448f-8b79-d23423ae8f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:21.508724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd70d1-eaad-448f-8b79-d23423ae8f06 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=9b6af8e4ca3048539f554280c06818af, user.id=dfc790aa3d3f4b8ead34a882e6d88224) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:21.509032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd70d1-eaad-448f-8b79-d23423ae8f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:21.510501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd70d1-eaad-448f-8b79-d23423ae8f06 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:21.516287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 572/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:39:21 2022] GET /identity/v3/role_assignments?user.id=dfc790aa3d3f4b8ead34a882e6d88224&scope.project.id=9b6af8e4ca3048539f554280c06818af&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:23.345535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aecbc-d019-48fb-a3f9-95155e967c17 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:23.345992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aecbc-d019-48fb-a3f9-95155e967c17 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:23.346081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aecbc-d019-48fb-a3f9-95155e967c17 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:23.347410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 575/1147] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:23.352967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b325120-8789-43a2-898d-cce08a9257f8 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:23.353805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b325120-8789-43a2-898d-cce08a9257f8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:23.354086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b325120-8789-43a2-898d-cce08a9257f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:23.376751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5b325120-8789-43a2-898d-cce08a9257f8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:23.377928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5b325120-8789-43a2-898d-cce08a9257f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:23.452942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 573/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:23.721766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aecbc-d019-48fb-a3f9-95155e967c17 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:23.755062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:23.837446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aecbc-d019-48fb-a3f9-95155e967c17 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:23.874107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:23.874710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:23.874985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:23.875254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:23.886470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:39:23.891119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:39:23.892180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:39:23.893089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 576/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:39:23.899169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b325120-8789-43a2-898d-cce08a9257f8 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:23.994754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:24.002026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b325120-8789-43a2-898d-cce08a9257f8 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:24.041717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:24.042308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:24.042579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:24.042841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:24.043485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:24.043777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:24.045191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:24.053384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 574/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:24.059634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:24.146007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:24.164048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734221e-653d-4a29-9c65-18e4d486b2b2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:24.200657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:24.201261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:24.201528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:24.201793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] PATH_INFO: `/v3/users/dfc790aa3d3f4b8ead34a882e6d88224` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:24.203008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] RBAC: Authorizing `identity:get_user(user_id=dfc790aa3d3f4b8ead34a882e6d88224)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:24.204468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:24.206011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 577/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:24 2022] GET /identity/v3/users/dfc790aa3d3f4b8ead34a882e6d88224 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:24.214769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:24.326074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d3839-be5e-4f01-9ee9-68d867318d94 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:24.361732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40680f3-d29a-49f9-a8e9-2570ac616105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:24.362362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40680f3-d29a-49f9-a8e9-2570ac616105 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:24.362623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40680f3-d29a-49f9-a8e9-2570ac616105 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:24.362894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40680f3-d29a-49f9-a8e9-2570ac616105 None admin] PATH_INFO: `/v3/projects/9b6af8e4ca3048539f554280c06818af` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:24.364022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40680f3-d29a-49f9-a8e9-2570ac616105 None admin] RBAC: Authorizing `identity:get_project(project_id=9b6af8e4ca3048539f554280c06818af)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:24.365508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40680f3-d29a-49f9-a8e9-2570ac616105 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:24.367143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 575/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:24 2022] GET /identity/v3/projects/9b6af8e4ca3048539f554280c06818af => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:24.373755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:24.481100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f370-b20a-4f9f-bbf3-3994f7e4542d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:24.516010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93901ff-8a82-482a-9fbe-6b1ceaf9d4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:24.516676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93901ff-8a82-482a-9fbe-6b1ceaf9d4b1 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:24.516928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93901ff-8a82-482a-9fbe-6b1ceaf9d4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:24.517195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93901ff-8a82-482a-9fbe-6b1ceaf9d4b1 None admin] PATH_INFO: `/v3/projects/9b6af8e4ca3048539f554280c06818af/users/dfc790aa3d3f4b8ead34a882e6d88224/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:24.519117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93901ff-8a82-482a-9fbe-6b1ceaf9d4b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b6af8e4ca3048539f554280c06818af, user_id=dfc790aa3d3f4b8ead34a882e6d88224, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:24.520685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93901ff-8a82-482a-9fbe-6b1ceaf9d4b1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:24.540767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 578/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:39:24 2022] PUT /identity/v3/projects/9b6af8e4ca3048539f554280c06818af/users/dfc790aa3d3f4b8ead34a882e6d88224/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:39:26.377321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab7a61-917d-42d9-a893-8b53323ec9c4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:26.378102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab7a61-917d-42d9-a893-8b53323ec9c4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:26.378102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab7a61-917d-42d9-a893-8b53323ec9c4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:26.380333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 576/1154] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:26.387039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc298d-0c83-4d3a-93e7-fe0a300c400f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:26.387749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc298d-0c83-4d3a-93e7-fe0a300c400f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:26.387932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc298d-0c83-4d3a-93e7-fe0a300c400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:26.410829 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-76bc298d-0c83-4d3a-93e7-fe0a300c400f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:26.412021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-76bc298d-0c83-4d3a-93e7-fe0a300c400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:26.421232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:26.484401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 579/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:26.754128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab7a61-917d-42d9-a893-8b53323ec9c4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:26.784571 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:26.880284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab7a61-917d-42d9-a893-8b53323ec9c4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:26.919943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:26.920584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:26.920855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:26.921120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:26.931937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:39:26.936538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:39:26.939646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:39:26.940836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 577/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:39:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:39:26.944326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc298d-0c83-4d3a-93e7-fe0a300c400f None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:27.075835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc298d-0c83-4d3a-93e7-fe0a300c400f None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:27.121555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:27.122139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:27.122407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:27.122676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:27.123336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:27.123624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:27.125036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:27.134415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 580/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:39:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:27.143917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:27.234515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:27.251772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded35fe0-9fea-4e0e-bf78-6f25e4a1912f None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:27.288070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:27.288722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:27.288992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:27.289247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] PATH_INFO: `/v3/users/dfc790aa3d3f4b8ead34a882e6d88224` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:27.290412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] RBAC: Authorizing `identity:get_user(user_id=dfc790aa3d3f4b8ead34a882e6d88224)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:27.291886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:27.293465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 578/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:27 2022] GET /identity/v3/users/dfc790aa3d3f4b8ead34a882e6d88224 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:27.299782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:27.309745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:27.408422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e34db7-109c-416b-9fa6-deabacb9770d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:27.443286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:27.443872 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:27.444146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:27.444442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:27.517237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:27.518695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:27.526051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:27.627490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 581/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:39:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:39:27.633594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:27.757966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80027a8-d3bb-4afe-b6ef-a14d6fb04a01 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:27.795216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3050995-cc62-4876-a9be-b16cd0c7db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:27.795827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050995-cc62-4876-a9be-b16cd0c7db41 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:27.796098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050995-cc62-4876-a9be-b16cd0c7db41 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:27.796391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050995-cc62-4876-a9be-b16cd0c7db41 None admin] PATH_INFO: `/v3/projects/9b6af8e4ca3048539f554280c06818af` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:27.797558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3050995-cc62-4876-a9be-b16cd0c7db41 None admin] RBAC: Authorizing `identity:get_project(project_id=9b6af8e4ca3048539f554280c06818af)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:27.798997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3050995-cc62-4876-a9be-b16cd0c7db41 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:27.800657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 579/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:27 2022] GET /identity/v3/projects/9b6af8e4ca3048539f554280c06818af => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:27.810848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:27.926211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377d19d-7cc7-449f-9a0d-833b9636f1b2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:27.935179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:27.964607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e0383-166c-4f3d-aefa-d75b52dc9822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:27.965195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e0383-166c-4f3d-aefa-d75b52dc9822 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:27.965462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e0383-166c-4f3d-aefa-d75b52dc9822 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:27.965724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e0383-166c-4f3d-aefa-d75b52dc9822 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:27.966433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e0383-166c-4f3d-aefa-d75b52dc9822 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=9b6af8e4ca3048539f554280c06818af, user.id=dfc790aa3d3f4b8ead34a882e6d88224) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:27.966725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e0383-166c-4f3d-aefa-d75b52dc9822 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:27.968144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e0383-166c-4f3d-aefa-d75b52dc9822 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:27.973917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 582/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:39:27 2022] GET /identity/v3/role_assignments?user.id=dfc790aa3d3f4b8ead34a882e6d88224&scope.project.id=9b6af8e4ca3048539f554280c06818af&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:29.872480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac8b78-b021-417d-9ddf-c5ad3edd3dc5 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:29.873059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac8b78-b021-417d-9ddf-c5ad3edd3dc5 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:29.873059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac8b78-b021-417d-9ddf-c5ad3edd3dc5 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:29.874373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 580/1162] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:29.879781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e70b1-3d95-4fbc-ac2c-882139682b40 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:29.880221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e70b1-3d95-4fbc-ac2c-882139682b40 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:29.880334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e70b1-3d95-4fbc-ac2c-882139682b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:29.900293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-388e70b1-3d95-4fbc-ac2c-882139682b40 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:29.901475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-388e70b1-3d95-4fbc-ac2c-882139682b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:29.979726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 583/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:30.245475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac8b78-b021-417d-9ddf-c5ad3edd3dc5 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:30.275097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:30.362704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac8b78-b021-417d-9ddf-c5ad3edd3dc5 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:30.401192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:30.401778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:30.402049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:30.402311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:30.470816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:30.472297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:30.492583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:30.574225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 581/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:39:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:39:30.583705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e70b1-3d95-4fbc-ac2c-882139682b40 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:30.705974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:30.709774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e70b1-3d95-4fbc-ac2c-882139682b40 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:30.746853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54d053-1aec-40af-b299-13c90426d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:30.747466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54d053-1aec-40af-b299-13c90426d3bb None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:30.747736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54d053-1aec-40af-b299-13c90426d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:30.748005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54d053-1aec-40af-b299-13c90426d3bb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:30.757315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54d053-1aec-40af-b299-13c90426d3bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:30.758763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54d053-1aec-40af-b299-13c90426d3bb None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-de54d053-1aec-40af-b299-13c90426d3bb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:39:30.770809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:39:30.771977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:39:30.772905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 584/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:39:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:39:30.778864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:30.881762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c8702-fa2c-4804-a95a-375c0e16c710 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:30.909738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:30.925868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1305e4-b8ea-4e3c-8a0e-7b0fb563b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:30.926464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1305e4-b8ea-4e3c-8a0e-7b0fb563b531 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:30.926735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1305e4-b8ea-4e3c-8a0e-7b0fb563b531 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:30.927001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1305e4-b8ea-4e3c-8a0e-7b0fb563b531 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:30.927630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1305e4-b8ea-4e3c-8a0e-7b0fb563b531 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:30.927925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1305e4-b8ea-4e3c-8a0e-7b0fb563b531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:30.929330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1305e4-b8ea-4e3c-8a0e-7b0fb563b531 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:30.936734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 582/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Aug 12 10:39:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:32.792669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba111c8-973c-488b-817b-1f5a24a29ae0 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:32.793568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba111c8-973c-488b-817b-1f5a24a29ae0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:32.793568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba111c8-973c-488b-817b-1f5a24a29ae0 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:32.795655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 585/1167] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:32.801098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90708-79c4-433c-a1c7-5cd6209392b3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:32.801510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90708-79c4-433c-a1c7-5cd6209392b3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:32.801616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90708-79c4-433c-a1c7-5cd6209392b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:32.820992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e0d90708-79c4-433c-a1c7-5cd6209392b3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:32.822179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e0d90708-79c4-433c-a1c7-5cd6209392b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:32.909232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 583/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:33.180945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba111c8-973c-488b-817b-1f5a24a29ae0 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:33.290201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba111c8-973c-488b-817b-1f5a24a29ae0 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:33.328432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb86ba-6b67-43f1-84ea-d7b7877cc2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:33.329008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb86ba-6b67-43f1-84ea-d7b7877cc2ba None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:33.329274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb86ba-6b67-43f1-84ea-d7b7877cc2ba None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:33.329536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb86ba-6b67-43f1-84ea-d7b7877cc2ba None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:33.330184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb86ba-6b67-43f1-84ea-d7b7877cc2ba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:33.331535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb86ba-6b67-43f1-84ea-d7b7877cc2ba None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:33.350793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 586/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Fri Aug 12 10:39:33 2022] POST /identity/v3/domains => generated 267 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:39:35.299010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f69b0-7e9f-4a76-8eba-db359c32b9f7 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:35.299924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f69b0-7e9f-4a76-8eba-db359c32b9f7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:35.299924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f69b0-7e9f-4a76-8eba-db359c32b9f7 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:35.302152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 584/1170] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:35.308441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b8d1-7045-446e-a597-3e52d2c3393c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:35.309128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b8d1-7045-446e-a597-3e52d2c3393c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:35.309273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b8d1-7045-446e-a597-3e52d2c3393c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:35.338103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f041b8d1-7045-446e-a597-3e52d2c3393c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:35.339410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f041b8d1-7045-446e-a597-3e52d2c3393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:35.354555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:35.411728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 587/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:35.678820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f69b0-7e9f-4a76-8eba-db359c32b9f7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:35.796747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f69b0-7e9f-4a76-8eba-db359c32b9f7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:35.810804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:35.834050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c69389-cb80-4b12-8dc1-f0e35a15d7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:35.834959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69389-cb80-4b12-8dc1-f0e35a15d7c3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:35.835355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69389-cb80-4b12-8dc1-f0e35a15d7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:35.835708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c69389-cb80-4b12-8dc1-f0e35a15d7c3 None admin] PATH_INFO: `/v3/domains/126f1eb3cc6a47f2b197005517f9e5fc` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:35.847332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c69389-cb80-4b12-8dc1-f0e35a15d7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=126f1eb3cc6a47f2b197005517f9e5fc)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:35.848811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c69389-cb80-4b12-8dc1-f0e35a15d7c3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:35.850374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 585/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Fri Aug 12 10:39:35 2022] GET /identity/v3/domains/126f1eb3cc6a47f2b197005517f9e5fc => generated 267 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:35.856832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b8d1-7045-446e-a597-3e52d2c3393c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:35.963698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b8d1-7045-446e-a597-3e52d2c3393c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:36.000208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80401a07-a5d5-4b17-8cb6-b066bc6f2a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:36.001118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80401a07-a5d5-4b17-8cb6-b066bc6f2a49 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:36.001476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80401a07-a5d5-4b17-8cb6-b066bc6f2a49 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:36.001831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80401a07-a5d5-4b17-8cb6-b066bc6f2a49 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:36.002801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80401a07-a5d5-4b17-8cb6-b066bc6f2a49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:36.005115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80401a07-a5d5-4b17-8cb6-b066bc6f2a49 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:36.041602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 588/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:39:35 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:39:37.919483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e8796-a0ca-4666-906a-4d8873f6be3e None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:37.920069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e8796-a0ca-4666-906a-4d8873f6be3e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:37.920069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e8796-a0ca-4666-906a-4d8873f6be3e None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:37.921378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 586/1174] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:37.927784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ef170-6c4c-41da-9ff1-a723ea7c0479 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:37.928539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ef170-6c4c-41da-9ff1-a723ea7c0479 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:37.928700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ef170-6c4c-41da-9ff1-a723ea7c0479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:37.936601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:37.951202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a83ef170-6c4c-41da-9ff1-a723ea7c0479 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:37.952403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a83ef170-6c4c-41da-9ff1-a723ea7c0479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:38.021699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 589/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:38.282300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e8796-a0ca-4666-906a-4d8873f6be3e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:38.389604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e8796-a0ca-4666-906a-4d8873f6be3e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:38.430279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065011a-d4e6-4e07-a26b-f0f890ee8da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:38.431017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065011a-d4e6-4e07-a26b-f0f890ee8da7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:38.431551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065011a-d4e6-4e07-a26b-f0f890ee8da7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:38.431551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065011a-d4e6-4e07-a26b-f0f890ee8da7 None admin] PATH_INFO: `/v3/domains/126f1eb3cc6a47f2b197005517f9e5fc` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:38.432730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065011a-d4e6-4e07-a26b-f0f890ee8da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=126f1eb3cc6a47f2b197005517f9e5fc)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:38.434179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065011a-d4e6-4e07-a26b-f0f890ee8da7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:38.435745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 587/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Fri Aug 12 10:39:38 2022] GET /identity/v3/domains/126f1eb3cc6a47f2b197005517f9e5fc => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:38.441967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ef170-6c4c-41da-9ff1-a723ea7c0479 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:38.556021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ef170-6c4c-41da-9ff1-a723ea7c0479 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:38.587309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:38.593560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def93de5-a2e4-441e-ba90-dbd476e26702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:38.594157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93de5-a2e4-441e-ba90-dbd476e26702 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:38.594434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93de5-a2e4-441e-ba90-dbd476e26702 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:38.594698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93de5-a2e4-441e-ba90-dbd476e26702 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:38.595339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93de5-a2e4-441e-ba90-dbd476e26702 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:38.596805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93de5-a2e4-441e-ba90-dbd476e26702 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:38.619100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 590/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:39:38 2022] POST /identity/v3/users => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:39:40.536663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044187e-27c5-4077-93b7-1db2d1e6bd65 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:40.537531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044187e-27c5-4077-93b7-1db2d1e6bd65 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:40.537531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044187e-27c5-4077-93b7-1db2d1e6bd65 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:40.539670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 588/1178] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:40.548503 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ab89d-4e2f-446e-8bba-bfbe00c41515 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:40.548896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ab89d-4e2f-446e-8bba-bfbe00c41515 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:40.549044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ab89d-4e2f-446e-8bba-bfbe00c41515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:40.571517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ec3ab89d-4e2f-446e-8bba-bfbe00c41515 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:40.572723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ec3ab89d-4e2f-446e-8bba-bfbe00c41515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:40.641128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 591/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:40.905199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044187e-27c5-4077-93b7-1db2d1e6bd65 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:40.955996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:41.024457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044187e-27c5-4077-93b7-1db2d1e6bd65 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:41.062233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:41.062857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:41.063127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:41.063383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:41.074754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:39:41.079297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:39:41.080530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:39:41.081883 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 589/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:39:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:39:41.086955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ab89d-4e2f-446e-8bba-bfbe00c41515 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:41.198025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ab89d-4e2f-446e-8bba-bfbe00c41515 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:41.221458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:41.239988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:41.240611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:41.240884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:41.241145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:41.241801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:41.242092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:41.243979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:41.252247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 592/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:39:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:41.258727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:41.373676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a0b58-b853-4acc-80b9-b54d2b8dab9d None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:41.375251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:41.411139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:41.411743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:41.412014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:41.412302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] PATH_INFO: `/v3/users/1f45fc2b14ff476abe26ff38bd045212` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:41.425484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] RBAC: Authorizing `identity:get_user(user_id=1f45fc2b14ff476abe26ff38bd045212)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:41.426934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:41.428534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 590/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:41 2022] GET /identity/v3/users/1f45fc2b14ff476abe26ff38bd045212 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:41.434985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:41.544420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25380-3dd9-4eb8-a206-241b9970dae6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:41.579684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:41.580299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:41.580550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:41.580818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:41.628802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:41.648485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:41.650018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:41.749777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 593/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:39:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:39:41.756091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:41.870181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb32-d7cd-4b88-b971-06dcfdeece74 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:41.905301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd58ca-7947-409d-80d5-0cbcc1cfa8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:41.905921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd58ca-7947-409d-80d5-0cbcc1cfa8fd None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:41.906194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd58ca-7947-409d-80d5-0cbcc1cfa8fd None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:41.906459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd58ca-7947-409d-80d5-0cbcc1cfa8fd None admin] PATH_INFO: `/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:41.907608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd58ca-7947-409d-80d5-0cbcc1cfa8fd None admin] RBAC: Authorizing `identity:get_project(project_id=104adfa5d7cd454ba8574ecdf1ffec7c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:41.909067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd58ca-7947-409d-80d5-0cbcc1cfa8fd None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:41.910682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 591/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:41 2022] GET /identity/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:41.918269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:42.004778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:42.024760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4685a-45f4-477f-b780-1714f199cfa8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:42.063620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73848f58-299a-4a1b-a136-7362a7095157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:42.064199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73848f58-299a-4a1b-a136-7362a7095157 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:42.064500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73848f58-299a-4a1b-a136-7362a7095157 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:42.064760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73848f58-299a-4a1b-a136-7362a7095157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:42.065470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73848f58-299a-4a1b-a136-7362a7095157 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=104adfa5d7cd454ba8574ecdf1ffec7c, user.id=1f45fc2b14ff476abe26ff38bd045212) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:42.065767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73848f58-299a-4a1b-a136-7362a7095157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:42.067189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73848f58-299a-4a1b-a136-7362a7095157 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:42.072445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 594/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:39:41 2022] GET /identity/v3/role_assignments?user.id=1f45fc2b14ff476abe26ff38bd045212&scope.project.id=104adfa5d7cd454ba8574ecdf1ffec7c&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:43.936776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c69c-77ef-4f13-86f8-368ea46f0516 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:43.937612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c69c-77ef-4f13-86f8-368ea46f0516 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:43.937612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c69c-77ef-4f13-86f8-368ea46f0516 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:43.939772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 592/1186] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:43.946199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39a4e0-0bf3-47f3-9104-2941ec6c87ce None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:43.947564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39a4e0-0bf3-47f3-9104-2941ec6c87ce None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:43.948051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39a4e0-0bf3-47f3-9104-2941ec6c87ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:43.969696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ce39a4e0-0bf3-47f3-9104-2941ec6c87ce None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:43.970902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ce39a4e0-0bf3-47f3-9104-2941ec6c87ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:44.041981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 595/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:44.311531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c69c-77ef-4f13-86f8-368ea46f0516 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:44.350860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:44.434240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c69c-77ef-4f13-86f8-368ea46f0516 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:44.471256 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:44.471862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:44.472132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:44.472426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:44.483586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:39:44.488146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:39:44.489331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:39:44.490184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 593/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:39:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:39:44.495601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39a4e0-0bf3-47f3-9104-2941ec6c87ce None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:44.578394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:44.604409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39a4e0-0bf3-47f3-9104-2941ec6c87ce None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:44.637745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:44.638336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:44.638611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:44.638875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:44.639532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:44.639825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:44.641233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:44.649143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 596/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:39:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:44.658438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:44.752650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:44.773159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2090d6-2a89-4d0e-93f6-3c5bc866c963 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:44.817725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:44.818348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:44.818620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:44.818884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] PATH_INFO: `/v3/users/1f45fc2b14ff476abe26ff38bd045212` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:44.820232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] RBAC: Authorizing `identity:get_user(user_id=1f45fc2b14ff476abe26ff38bd045212)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:44.821684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:44.823283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 594/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:44 2022] GET /identity/v3/users/1f45fc2b14ff476abe26ff38bd045212 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:44.830775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:44.941439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cf90e-c200-4b73-8818-790cf290dadc None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:44.981372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c78947-b0d9-40e2-8a31-fbe9d236896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:44.981968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c78947-b0d9-40e2-8a31-fbe9d236896a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:44.982243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c78947-b0d9-40e2-8a31-fbe9d236896a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:44.982510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c78947-b0d9-40e2-8a31-fbe9d236896a None admin] PATH_INFO: `/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:44.983661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c78947-b0d9-40e2-8a31-fbe9d236896a None admin] RBAC: Authorizing `identity:get_project(project_id=104adfa5d7cd454ba8574ecdf1ffec7c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:44.985135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c78947-b0d9-40e2-8a31-fbe9d236896a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:44.986739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 597/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:44 2022] GET /identity/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:44.993089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:45.117923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0619e-8adb-4a37-bdbd-18768d3bd0d6 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:45.161872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da152d-2aa7-44b2-836d-4b89b90dcf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:45.162806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da152d-2aa7-44b2-836d-4b89b90dcf01 None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:45.163210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da152d-2aa7-44b2-836d-4b89b90dcf01 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:45.163609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da152d-2aa7-44b2-836d-4b89b90dcf01 None admin] PATH_INFO: `/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c/users/1f45fc2b14ff476abe26ff38bd045212/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:45.166543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da152d-2aa7-44b2-836d-4b89b90dcf01 None admin] RBAC: Authorizing `identity:create_grant(project_id=104adfa5d7cd454ba8574ecdf1ffec7c, user_id=1f45fc2b14ff476abe26ff38bd045212, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:45.168824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da152d-2aa7-44b2-836d-4b89b90dcf01 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:45.186467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 595/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:39:44 2022] PUT /identity/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c/users/1f45fc2b14ff476abe26ff38bd045212/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:39:47.079750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c15b30-436c-4b62-b8b1-6647dcc0ca76 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:47.080366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c15b30-436c-4b62-b8b1-6647dcc0ca76 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:47.080366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c15b30-436c-4b62-b8b1-6647dcc0ca76 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:47.081652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 598/1193] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:39:47.087082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510e28c-b222-4c79-ba3e-dc768da40286 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:47.087516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510e28c-b222-4c79-ba3e-dc768da40286 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:47.087616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510e28c-b222-4c79-ba3e-dc768da40286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:47.107147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b510e28c-b222-4c79-ba3e-dc768da40286 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:39:47.108305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b510e28c-b222-4c79-ba3e-dc768da40286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:39:47.119842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:47.187404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 596/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:39:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:39:47.451351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c15b30-436c-4b62-b8b1-6647dcc0ca76 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:47.458029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:47.567745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c15b30-436c-4b62-b8b1-6647dcc0ca76 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:47.602778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:47.603300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:47.603572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:47.603834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:47.615211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:39:47.619860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:39:47.620963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:39:47.621946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 599/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:39:47.627950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510e28c-b222-4c79-ba3e-dc768da40286 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:47.731346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510e28c-b222-4c79-ba3e-dc768da40286 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:47.766304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:47.766898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:47.767161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:47.767438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:47.768058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:47.768379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:47.769734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:47.777853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 597/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:47.787292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:47.861902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:47.901357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ee846-e8f3-4210-9db6-9968d4bbbb78 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:47.934892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:47.935491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:47.935760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:47.936022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] PATH_INFO: `/v3/users/1f45fc2b14ff476abe26ff38bd045212` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:47.937199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] RBAC: Authorizing `identity:get_user(user_id=1f45fc2b14ff476abe26ff38bd045212)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:47.938629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:47.940184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 600/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Aug 12 10:39:47 2022] GET /identity/v3/users/1f45fc2b14ff476abe26ff38bd045212 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:47.946439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:47.958370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:48.048676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bb14d-67ef-4f50-af59-9c05bb971f99 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:48.082457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:48.083052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:48.083319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:48.083580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:48.162317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:48.163771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:48.173419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:48.262577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 598/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:39:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:39:48.268869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:48.381197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b037e5-594e-452c-81b2-754cb697242e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:48.410751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:48.417010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4221c8c-1157-4059-92da-e42e73e626bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:48.417558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221c8c-1157-4059-92da-e42e73e626bd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:48.417832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221c8c-1157-4059-92da-e42e73e626bd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:48.418098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221c8c-1157-4059-92da-e42e73e626bd None admin] PATH_INFO: `/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:48.419256 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4221c8c-1157-4059-92da-e42e73e626bd None admin] RBAC: Authorizing `identity:get_project(project_id=104adfa5d7cd454ba8574ecdf1ffec7c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:48.420732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4221c8c-1157-4059-92da-e42e73e626bd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:48.422351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 601/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Aug 12 10:39:48 2022] GET /identity/v3/projects/104adfa5d7cd454ba8574ecdf1ffec7c => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:48.428959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:39:48.544705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf2a1c-368b-4159-9190-efa6a481c6d3 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:39:48.553966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:39:48.583640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f85f5-0dde-4336-96df-e4a4daa5decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:39:48.584238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f85f5-0dde-4336-96df-e4a4daa5decc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:39:48.584536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f85f5-0dde-4336-96df-e4a4daa5decc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:39:48.584800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f85f5-0dde-4336-96df-e4a4daa5decc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:39:48.585517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f85f5-0dde-4336-96df-e4a4daa5decc None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=104adfa5d7cd454ba8574ecdf1ffec7c, user.id=1f45fc2b14ff476abe26ff38bd045212) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:39:48.585916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f85f5-0dde-4336-96df-e4a4daa5decc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:39:48.587811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f85f5-0dde-4336-96df-e4a4daa5decc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:39:48.593912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 599/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:39:48 2022] GET /identity/v3/role_assignments?user.id=1f45fc2b14ff476abe26ff38bd045212&scope.project.id=104adfa5d7cd454ba8574ecdf1ffec7c&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:39:58.592799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=14 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:43.814621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068386-7b25-42df-918d-34df8e9edd87 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:43.815231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068386-7b25-42df-918d-34df8e9edd87 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:43.815231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068386-7b25-42df-918d-34df8e9edd87 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:43.816666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 602/1201] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:40:43.823364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875d6a4-cea8-46e0-890d-eaeefe3594a4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:43.823964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875d6a4-cea8-46e0-890d-eaeefe3594a4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:43.823964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875d6a4-cea8-46e0-890d-eaeefe3594a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:43.846066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2875d6a4-cea8-46e0-890d-eaeefe3594a4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:40:43.847584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2875d6a4-cea8-46e0-890d-eaeefe3594a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:40:43.931822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 600/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:40:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:40:44.218614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a068386-7b25-42df-918d-34df8e9edd87 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:44.355057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a068386-7b25-42df-918d-34df8e9edd87 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:44.399733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:44.400398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:44.400747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:44.401017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:44.413514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:44.415148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:40:44.432577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:44.433892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:40:44.435004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:40:44.435004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:44.435004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:40:44.435004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 603/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:44.440576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875d6a4-cea8-46e0-890d-eaeefe3594a4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:44.567650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875d6a4-cea8-46e0-890d-eaeefe3594a4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:44.602416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:44.610677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a17bf-579a-4277-a875-2f534fe543b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:44.611392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a17bf-579a-4277-a875-2f534fe543b7 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:44.611700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a17bf-579a-4277-a875-2f534fe543b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:44.612036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a17bf-579a-4277-a875-2f534fe543b7 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:44.612810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a17bf-579a-4277-a875-2f534fe543b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:44.613231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a17bf-579a-4277-a875-2f534fe543b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:44.614847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a17bf-579a-4277-a875-2f534fe543b7 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:44.636367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 601/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:44.642852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:44.706722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:44.763179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aa998-7929-4cb1-a43a-d54b3c827b50 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:44.809193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10828a34-446e-4af0-b241-e2419cd9e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:44.809802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828a34-446e-4af0-b241-e2419cd9e902 None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:44.810162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828a34-446e-4af0-b241-e2419cd9e902 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:44.810384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828a34-446e-4af0-b241-e2419cd9e902 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:44.810996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10828a34-446e-4af0-b241-e2419cd9e902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:44.812475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10828a34-446e-4af0-b241-e2419cd9e902 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:44.840376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 604/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Aug 12 10:40:44 2022] POST /identity/v3/users => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:40:46.938098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc716e9-bccb-40bf-a983-18b66a6d9916 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:46.938650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc716e9-bccb-40bf-a983-18b66a6d9916 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:46.938650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc716e9-bccb-40bf-a983-18b66a6d9916 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:46.939985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 602/1206] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:40:46.945329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f256525-4dbc-421a-9c5e-23016311e116 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:46.945891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f256525-4dbc-421a-9c5e-23016311e116 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:46.946125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f256525-4dbc-421a-9c5e-23016311e116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:46.965170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3f256525-4dbc-421a-9c5e-23016311e116 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:40:46.966378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3f256525-4dbc-421a-9c5e-23016311e116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:40:47.044308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 605/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:40:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:40:47.305077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc716e9-bccb-40bf-a983-18b66a6d9916 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:47.428851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc716e9-bccb-40bf-a983-18b66a6d9916 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:47.467582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060255-1630-440f-a575-b3a189234601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:47.468322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060255-1630-440f-a575-b3a189234601 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:47.468563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060255-1630-440f-a575-b3a189234601 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:47.468854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060255-1630-440f-a575-b3a189234601 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:47.480123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be060255-1630-440f-a575-b3a189234601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-be060255-1630-440f-a575-b3a189234601 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:40:47.484920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:47.486062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:47.487187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 603/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:40:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:47.492167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f256525-4dbc-421a-9c5e-23016311e116 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:47.535439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:47.604586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f256525-4dbc-421a-9c5e-23016311e116 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:47.642756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:47.643359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:47.643638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:47.643907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:47.644614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:47.644914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:47.646298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:47.656099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 606/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:40:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:47.661859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060255-1630-440f-a575-b3a189234601 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:47.719033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:47.784041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060255-1630-440f-a575-b3a189234601 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:47.820242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:47.821013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:47.821704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:47.821819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:47.832895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:47.834755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:40:47.852861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:47.854030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:40:47.855186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:40:47.855186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:47.855186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:47.856611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 604/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:47.863335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:47.972032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:47.975404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aec240-70b9-4fe4-9793-3027f19d20fe None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:48.014466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:48.015112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:48.015378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:48.015771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:48.016469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:48.016785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:48.018300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:48.032062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 607/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:48.040548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:48.150046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1ca2a-347b-4552-8272-fccf788f67a1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:48.151586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:48.185282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:48.185913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:48.186185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:48.186493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:48.194774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:48.196486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:40:48.209202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:48.210174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:40:48.211161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:48.213941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 605/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:40:48.219752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:48.355708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd30887-1e97-463f-83ce-b780ecddb45a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:48.397109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:48.397719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:48.397994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:48.398262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:48.399049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:48.399481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:48.401031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:48.420044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 608/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Aug 12 10:40:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:48.427211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:48.549882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ae2f2-ba02-4c0e-8bac-57699aa9389b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:48.597398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:48.597993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:48.598304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:48.598546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:48.636994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:48.676862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:48.678408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:48.796473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 606/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:40:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:40:48.802102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:48.822692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:48.913288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc988de-ffde-4e80-a3ec-d409de3cdbe7 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:48.950673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:48.951227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:48.951501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:48.951917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:48.962594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:48.964055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:40:48.979231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:48.982116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:40:48.983041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:40:48.983041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:48.983041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:40:48.983041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 609/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:48.986431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:49.058664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:49.099118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc76452-2d3d-49c6-930c-a31138aa49d1 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:49.134723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:49.135350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:49.135937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:49.136206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:49.138379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:49.138703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:49.140097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:49.155423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 607/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:49.161389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:49.238793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:49.277221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf56f2-1ced-4d28-bed4-5061dd7dc7b9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:49.317652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:49.318356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:49.318565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:49.318815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:49.326075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:49.327500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:40:49.341810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:49.342774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:40:49.343729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:40:49.348301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 610/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:40:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:49.348929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:49.458341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb39d0-c86d-462b-94c3-1a55bf0586b5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:49.483080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:49.495975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a556034-50a2-4751-99ef-180a1dd7ff67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:49.496574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a556034-50a2-4751-99ef-180a1dd7ff67 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:49.496851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a556034-50a2-4751-99ef-180a1dd7ff67 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:49.497118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a556034-50a2-4751-99ef-180a1dd7ff67 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:49.497804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a556034-50a2-4751-99ef-180a1dd7ff67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:49.498106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a556034-50a2-4751-99ef-180a1dd7ff67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:49.499558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a556034-50a2-4751-99ef-180a1dd7ff67 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:49.513017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 608/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:49.518978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:49.626575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ec8d-678e-4620-98af-6133f300cc5f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:49.658661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:49.666025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5788af3-eed7-4631-9082-997d90834cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:49.666707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788af3-eed7-4631-9082-997d90834cc3 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:49.666998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788af3-eed7-4631-9082-997d90834cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:49.667289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788af3-eed7-4631-9082-997d90834cc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:49.668057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5788af3-eed7-4631-9082-997d90834cc3 None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=fbb12187c53a40e79fe0227256346aeb) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:49.668475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5788af3-eed7-4631-9082-997d90834cc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:49.669966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5788af3-eed7-4631-9082-997d90834cc3 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:49.684301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 611/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:40:49 2022] GET /identity/v3/role_assignments?user.id=fbb12187c53a40e79fe0227256346aeb&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:51.712465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843a83b-b447-4d06-9997-15457d8951b6 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:51.713037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843a83b-b447-4d06-9997-15457d8951b6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:51.713037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843a83b-b447-4d06-9997-15457d8951b6 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:51.714383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 609/1220] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:40:51.719803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d5af5-1d42-41ea-8451-b848f42aa67e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:51.720248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d5af5-1d42-41ea-8451-b848f42aa67e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:51.720406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d5af5-1d42-41ea-8451-b848f42aa67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:51.741978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-219d5af5-1d42-41ea-8451-b848f42aa67e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:40:51.743445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-219d5af5-1d42-41ea-8451-b848f42aa67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:40:51.825525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 612/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:40:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:40:52.086365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843a83b-b447-4d06-9997-15457d8951b6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:52.205292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843a83b-b447-4d06-9997-15457d8951b6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:52.243952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:52.244616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:52.244895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:52.245162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:52.256665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:40:52.262212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:52.263356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:52.264492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 610/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:40:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:40:52.271351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d5af5-1d42-41ea-8451-b848f42aa67e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:52.381024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d5af5-1d42-41ea-8451-b848f42aa67e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:52.407269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:52.422868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:52.424200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:52.424746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:52.425027 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:52.425690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] RBAC: Adding query filter params (name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:52.425989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:52.427605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:52.443536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 613/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:40:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:52.444839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:52.496107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:52.557851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7beba-fcb6-4504-bb51-937196839432 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:52.594239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:52.594789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:52.595066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:52.595332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:52.606334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:52.608016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:40:52.625911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:52.632424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:40:52.633703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:40:52.633703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:52.633703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:52.633703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 611/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:52.634054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:52.743588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738f2d0-060a-4af7-a637-da5e9d32b86a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:52.778338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:52.778973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:52.779243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:52.779514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:52.780235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:52.780486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:52.781874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:52.794362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 614/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:52.801186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:52.905906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:52.912603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cb1c9-dc18-4537-8594-f6c44290ef49 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:52.950642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:52.951258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:52.951527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:52.951843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:52.959791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:52.961342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:40:52.974601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:52.981090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:52.982278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 612/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:52.982962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:53.032188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:53.093309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e90f-5194-4400-b08f-d4da06a61025 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:53.131424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:53.132021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:53.132319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:53.132585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:53.133347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:53.133645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:53.135078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:53.158189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 615/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Aug 12 10:40:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:53.159237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:53.260204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69d425-a3d2-4ab4-8de2-c234097f5c2b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:53.295946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:53.296609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:53.296885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:53.297148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:53.308306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:53.309792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:40:53.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:53.324669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:40:53.325793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:40:53.325793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:53.325793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:53.325793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 613/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:40:53.330823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:53.442531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a5b6-9231-4147-a923-bbc45d1693e8 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:53.444902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:53.480445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:53.481037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:53.481316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:53.481577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:53.482212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:53.482510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:53.483881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:53.495640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 616/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:53.501598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:53.511692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:53.610763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039175c6-ff05-4c62-bd63-dcb656fc18fb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:53.646966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:53.647615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:53.647889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:53.648151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:53.655636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:53.657296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:40:53.671467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:53.672634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:53.678578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 614/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:53.680357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:53.801095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219a333-9a54-47e2-8225-59477b2bf4ee None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:53.841154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1fe27-837c-43a2-93aa-6fbcfa1f7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:53.841753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1fe27-837c-43a2-93aa-6fbcfa1f7e94 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:53.842020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1fe27-837c-43a2-93aa-6fbcfa1f7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:53.842281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1fe27-837c-43a2-93aa-6fbcfa1f7e94 None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:53.842963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1fe27-837c-43a2-93aa-6fbcfa1f7e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:53.843258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1fe27-837c-43a2-93aa-6fbcfa1f7e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:53.844718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1fe27-837c-43a2-93aa-6fbcfa1f7e94 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:53.858338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 617/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:53.864824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:53.939360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:53.979888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c0319-0d65-47ac-960b-3d6917a1b090 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:54.019266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd98457-316a-4a07-8723-8a9023568e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:54.019980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd98457-316a-4a07-8723-8a9023568e9a None admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:54.020286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd98457-316a-4a07-8723-8a9023568e9a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:54.020589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd98457-316a-4a07-8723-8a9023568e9a None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/fbb12187c53a40e79fe0227256346aeb/roles/db7026479a164eec9e413091ad5440fe` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:54.038033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd98457-316a-4a07-8723-8a9023568e9a None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=fbb12187c53a40e79fe0227256346aeb, role_id=db7026479a164eec9e413091ad5440fe)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:54.039524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd98457-316a-4a07-8723-8a9023568e9a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:54.056050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 615/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:40:53 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/fbb12187c53a40e79fe0227256346aeb/roles/db7026479a164eec9e413091ad5440fe => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:40:56.011216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3397c-bfbd-417d-8332-31fef7717149 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:56.011782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3397c-bfbd-417d-8332-31fef7717149 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:56.011782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3397c-bfbd-417d-8332-31fef7717149 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:56.013423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 618/1233] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:40:56.018984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f83ed2-ebe6-48e2-b0a6-c71d304911bf None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:56.019369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f83ed2-ebe6-48e2-b0a6-c71d304911bf None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:56.019550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f83ed2-ebe6-48e2-b0a6-c71d304911bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:56.037637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d5f83ed2-ebe6-48e2-b0a6-c71d304911bf None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:40:56.038829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d5f83ed2-ebe6-48e2-b0a6-c71d304911bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:40:56.110409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 616/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:40:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:40:56.392662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3397c-bfbd-417d-8332-31fef7717149 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:56.418714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:56.499615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3397c-bfbd-417d-8332-31fef7717149 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:56.534359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:56.534983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:56.535248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:56.535511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:56.547475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:40:56.552125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:56.553286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 10:40:56.554413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:40:56.555376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 619/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Aug 12 10:40:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:56.559470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f83ed2-ebe6-48e2-b0a6-c71d304911bf None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:56.606854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:56.663237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f83ed2-ebe6-48e2-b0a6-c71d304911bf None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:56.697432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:56.698054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:56.698365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:56.698632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:56.699358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] RBAC: Adding query filter params (name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:56.699710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:56.701174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:56.715121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 617/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Aug 12 10:40:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:40:56.716513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:56.803964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:56.821803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6cc6-8214-493b-8d88-cf498c86fd8b None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:56.859929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:56.860519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:56.860796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:56.861062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:56.871431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:56.873040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:40:56.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:56.888425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:40:56.889442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:40:56.889442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:56.889442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:40:56.889442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 620/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:56.894535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:56.999467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4653cfb-7ab6-4e7a-b49c-37f613f4e058 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:57.008585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:57.033962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:57.034782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:57.034879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:57.035126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:57.035879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:57.036210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:57.037677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:57.054955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 618/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:57.061190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:57.170136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c547d0-3565-4740-ba80-0c3b2305851f None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:57.208152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:57.208796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:57.209061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:57.209324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:57.211938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:57.219095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:57.220611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:40:57.233568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:57.234769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:40:57.236130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 621/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:40:57.240755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:57.348002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a872fc8-3a6f-47b7-ac90-d7fab517f672 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:57.387506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:57.388115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:57.388434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:57.388687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:57.389410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:57.389722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:57.391113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:57.409438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 619/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Aug 12 10:40:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:57.415401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:57.522742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e2ee4-44ab-49f4-a8a1-723bc1ec80cd None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:57.556350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:57.556946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:57.557218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:57.557484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:57.619676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:57.626621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:57.628106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:57.747155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 622/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:40:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:40:57.753468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:57.785811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:57.863439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca7d2-7ee1-4a71-942a-54aefdd16a2a None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:57.900150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:57.900796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:57.901070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:57.901358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:57.912982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:57.914957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:40:57.930220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:57.931464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:40:57.932593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:40:57.932593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:40:57.932593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:57.932593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 620/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:57.940012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:58.057472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:58.064624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5cfcc-d012-46cf-b3c8-bebb45431895 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:58.104442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:58.105052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:58.105319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:58.105584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:58.106227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:58.106524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:58.107909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:58.119881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 623/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:40:58.125927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:58.214418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:40:58.240173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb72d2-c71e-43ec-aebd-d8f5e8503bf9 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:58.279042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:58.279774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:58.280207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:58.280591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:58.289808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:58.291326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:40:58.303385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:40:58.304443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:40:58.305411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 621/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:40:58.311640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:58.425169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309b6f1-08ee-4d10-8780-0cadbe91c9d6 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:58.462283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba0ef5-e781-4770-96bd-f3335958cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:58.462880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0ef5-e781-4770-96bd-f3335958cf7d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:58.463156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0ef5-e781-4770-96bd-f3335958cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:58.463427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0ef5-e781-4770-96bd-f3335958cf7d None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:58.464122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba0ef5-e781-4770-96bd-f3335958cf7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:58.464459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba0ef5-e781-4770-96bd-f3335958cf7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:58.465912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba0ef5-e781-4770-96bd-f3335958cf7d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:58.478482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 624/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:40:58.485382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:40:58.597407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3da77-0edb-4e5f-b6f7-b00438103deb None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:40:58.634321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c4b25-dfe5-4b3d-b80a-8f42195025fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:40:58.634940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c4b25-dfe5-4b3d-b80a-8f42195025fc None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:40:58.635232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c4b25-dfe5-4b3d-b80a-8f42195025fc None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:40:58.635486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c4b25-dfe5-4b3d-b80a-8f42195025fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:40:58.636553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c4b25-dfe5-4b3d-b80a-8f42195025fc None admin] RBAC: Adding query filter params (role.id=db7026479a164eec9e413091ad5440fe, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=fbb12187c53a40e79fe0227256346aeb) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:40:58.636857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c4b25-dfe5-4b3d-b80a-8f42195025fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:40:58.638307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c4b25-dfe5-4b3d-b80a-8f42195025fc None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:40:58.644594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 622/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:40:58 2022] GET /identity/v3/role_assignments?user.id=fbb12187c53a40e79fe0227256346aeb&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=db7026479a164eec9e413091ad5440fe => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:00.737780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319904be-ac6f-432b-aae4-33b6ff47551a None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:00.737780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319904be-ac6f-432b-aae4-33b6ff47551a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:00.737780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319904be-ac6f-432b-aae4-33b6ff47551a None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:00.740809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 625/1247] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:41:00.745468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac2171-6d53-435a-b2d5-1d356450697f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:00.745938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac2171-6d53-435a-b2d5-1d356450697f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:00.746029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac2171-6d53-435a-b2d5-1d356450697f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:00.752803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:00.766093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-14ac2171-6d53-435a-b2d5-1d356450697f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:41:00.767285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-14ac2171-6d53-435a-b2d5-1d356450697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:41:00.837104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 623/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:41:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 10:41:01.117878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319904be-ac6f-432b-aae4-33b6ff47551a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:01.140139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:01.229500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319904be-ac6f-432b-aae4-33b6ff47551a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:01.266388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:01.267013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:01.267287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:01.267555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:01.278358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:41:01.283310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:01.284424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:01.285527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 626/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:41:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:01.290932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac2171-6d53-435a-b2d5-1d356450697f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:01.416248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac2171-6d53-435a-b2d5-1d356450697f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:01.448293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:01.455761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:01.456487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:01.456807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:01.457318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:01.458033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:01.458339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:01.459726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:01.469648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 624/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:41:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:01.476481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:01.562156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:01.587295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e88ee-50c9-4ebb-a46e-20a7945e0dac None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:01.625473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:01.626093 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:01.626359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:01.626627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:01.640016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:01.642057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:41:01.659793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:01.660995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:41:01.662064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:41:01.662064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:01.662064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:01.662064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 627/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:01.668550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:01.794993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647c2e-7994-4c06-b48c-fedefe1ba99b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:01.831279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:01.831880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:01.832156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:01.832458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:01.833147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:01.833459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:01.834858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:01.848412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 625/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:01.854644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:01.966857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebbcc7-bb03-4fdc-be68-42de06771a98 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:01.997638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:02.005970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:02.006609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:02.006898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:02.007164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] PATH_INFO: `/v3/users/placement` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:02.015745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:02.017397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:41:02.029778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:02.031245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:02.032215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 628/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:02.037200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:02.101570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:02.153086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca17de4-88c0-46b3-aa95-6b8254e25bad None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:02.188410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:02.188993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:02.189291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:02.189554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:02.190272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:02.190567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:02.192215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:02.212302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 626/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Aug 12 10:41:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:41:02.217026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:02.323550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f44717-5f18-44b6-9e5d-ae4605aeae1d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:02.362844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:02.363439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:02.363715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:02.363988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:02.432222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:02.448169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:02.449793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:02.566614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 629/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:41:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 12 10:41:02.572825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:02.686126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd476a-7e87-46ac-ab4b-41926e3e5c07 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:02.701124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:02.729696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:02.730396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:02.730671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:02.730944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:02.742329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:02.743848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:41:02.759282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:02.760951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:41:02.762123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:41:02.762123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:02.762123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:02.762123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 627/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:02.766759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:02.866852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:02.880343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a299-2556-4c5c-9de8-93eaf887d048 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:02.918894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:02.919501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:02.919776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:02.920051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:02.920720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:02.921020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:02.922407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:02.934391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 630/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:41:02.940885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:03.062835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024a9a0-e8e4-46be-964f-31cfd3bb7e1e None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:03.100758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:03.101756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:03.103000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:03.103284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:03.115007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:03.117049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:41:03.130266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:03.131178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:03.132180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 628/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:41:03.137822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:03.251187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2442bc-8674-4885-b82f-3685cc7e449a None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:03.290871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07103f4a-51ba-49ed-9db0-b73b0898fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:03.291492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07103f4a-51ba-49ed-9db0-b73b0898fb9b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:03.291756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07103f4a-51ba-49ed-9db0-b73b0898fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:03.292016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07103f4a-51ba-49ed-9db0-b73b0898fb9b None admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:03.292728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07103f4a-51ba-49ed-9db0-b73b0898fb9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:03.293020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07103f4a-51ba-49ed-9db0-b73b0898fb9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:03.294444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07103f4a-51ba-49ed-9db0-b73b0898fb9b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:03.305970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 631/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:41:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:03.312109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:03.342832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:03.433217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab785f7-4b97-4ab5-994b-ec58166df038 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:03.472025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa26f1-132d-4308-9f0d-474b48eb92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:03.472700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa26f1-132d-4308-9f0d-474b48eb92d3 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:03.473040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa26f1-132d-4308-9f0d-474b48eb92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:03.473290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa26f1-132d-4308-9f0d-474b48eb92d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:03.474027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa26f1-132d-4308-9f0d-474b48eb92d3 None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=fbb12187c53a40e79fe0227256346aeb) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:03.474330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa26f1-132d-4308-9f0d-474b48eb92d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:03.475764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa26f1-132d-4308-9f0d-474b48eb92d3 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:03.482444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 629/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:41:03 2022] GET /identity/v3/role_assignments?user.id=fbb12187c53a40e79fe0227256346aeb&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=e65d667683404c438483a1b1252ff2bb => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:05.426379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e708-a384-449f-9023-24bb6e10de77 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:05.427248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e708-a384-449f-9023-24bb6e10de77 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:05.427248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1e708-a384-449f-9023-24bb6e10de77 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:05.429405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 632/1261] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:41:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:41:05.438303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35fabd-149e-4a0b-8ae9-ac7fff6b4b6a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:05.439047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35fabd-149e-4a0b-8ae9-ac7fff6b4b6a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:05.439229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35fabd-149e-4a0b-8ae9-ac7fff6b4b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:05.461983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-bc35fabd-149e-4a0b-8ae9-ac7fff6b4b6a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:41:05.463164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-bc35fabd-149e-4a0b-8ae9-ac7fff6b4b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:41:05.537013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 630/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:41:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:41:05.799346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1e708-a384-449f-9023-24bb6e10de77 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:05.812026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:05.925632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1e708-a384-449f-9023-24bb6e10de77 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:05.962185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:05.962806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:05.963080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:05.963346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:05.974726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:41:05.979443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:05.980512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:05.981490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 633/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:05.987110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35fabd-149e-4a0b-8ae9-ac7fff6b4b6a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:06.016137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:06.109254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35fabd-149e-4a0b-8ae9-ac7fff6b4b6a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:06.151966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:06.152599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:06.152874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:06.153167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:06.153828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:06.154126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:06.155502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:06.164620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 631/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:06.170745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:06.240935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:06.279922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a15189-224a-44ed-95d7-e9c518d3da94 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:06.315352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:06.316010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:06.316301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:06.316562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:06.327241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:06.328730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:41:06.343622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:06.344811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:41:06.345975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:41:06.345975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:06.345975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:06.345975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 634/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:06.350690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:06.434929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:06.457514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f0869-4f5c-44d5-aa10-16c71d7f8b93 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:06.495526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:06.496118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:06.496427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:06.496697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:06.497331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:06.497631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:06.498996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:06.513330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 632/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:06.525212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:06.642650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce16ab4-4db9-4517-bb15-a5e25bbe8ee9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:06.662857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:06.679657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:06.680307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:06.680587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:06.680880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:06.689213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:06.690602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:41:06.701724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:06.702731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:06.703744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 635/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:06.709108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:06.807995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72373ea7-8818-49ea-aa13-77504a7493b5 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:06.842300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:06.842880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:06.843147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:06.843408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:06.844116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:06.844503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:06.845919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:06.860650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:06.873448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 633/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Aug 12 10:41:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:06.879483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:06.980779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa7ac1-9976-4571-bb37-3bd0218c254b None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:07.016490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:07.017107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:07.017379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:07.017644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:07.031084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:07.032566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:41:07.052018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:07.053142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:41:07.054329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:41:07.054329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:07.054329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:07.054329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 636/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:07.060928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:07.177777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825a0d-a8e4-4579-a8f7-8a33867f5460 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:07.214052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:07.214648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:07.214917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:07.215185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:07.215817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:07.216118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:07.217523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:07.229071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 634/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:41:07.235297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:07.271302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:07.342321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0af05-e7b9-48c9-8fd0-ce9603c3bdf9 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:07.381025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:07.381629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:07.381903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:07.382165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:07.390280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:07.391730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:41:07.403332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:07.404405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:41:07.405310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:07.408491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 637/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:07.410352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:07.477725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:07.520673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a06c9f-99d8-4cbc-a2d3-fffe33a42867 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:07.554655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7ef61-647e-40c8-bcf3-67a9fe55f817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:07.555276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7ef61-647e-40c8-bcf3-67a9fe55f817 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:07.555618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7ef61-647e-40c8-bcf3-67a9fe55f817 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:07.555763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7ef61-647e-40c8-bcf3-67a9fe55f817 None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:07.556503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7ef61-647e-40c8-bcf3-67a9fe55f817 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:07.556796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7ef61-647e-40c8-bcf3-67a9fe55f817 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:07.558221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7ef61-647e-40c8-bcf3-67a9fe55f817 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:07.570012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 635/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:07.576426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:07.676730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:07.687313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee84aea-32c6-4757-8fe9-933f9c766571 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:07.722667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f2849-1e5a-42ea-bc05-c4cbd0d22fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:07.723297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f2849-1e5a-42ea-bc05-c4cbd0d22fa0 None admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:07.723568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f2849-1e5a-42ea-bc05-c4cbd0d22fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:07.723835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f2849-1e5a-42ea-bc05-c4cbd0d22fa0 None admin] PATH_INFO: `/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/fbb12187c53a40e79fe0227256346aeb/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:07.725858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f2849-1e5a-42ea-bc05-c4cbd0d22fa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e93cd13261f41b4971e1b0a6699a850, user_id=fbb12187c53a40e79fe0227256346aeb, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:07.727364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f2849-1e5a-42ea-bc05-c4cbd0d22fa0 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:07.745748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 638/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Aug 12 10:41:07 2022] PUT /identity/v3/projects/1e93cd13261f41b4971e1b0a6699a850/users/fbb12187c53a40e79fe0227256346aeb/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:41:09.649995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bbad6-0887-4a7e-b7a5-343010cbf5bb None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:09.650531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bbad6-0887-4a7e-b7a5-343010cbf5bb None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:09.650531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bbad6-0887-4a7e-b7a5-343010cbf5bb None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:09.651779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 636/1274] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:41:09.658074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403cfb3-c5bd-4791-af5e-647509c10dc1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:09.658842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403cfb3-c5bd-4791-af5e-647509c10dc1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:09.659008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403cfb3-c5bd-4791-af5e-647509c10dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:09.686156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2403cfb3-c5bd-4791-af5e-647509c10dc1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:41:09.687321 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2403cfb3-c5bd-4791-af5e-647509c10dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:41:09.757951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 639/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:41:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:41:10.038361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bbad6-0887-4a7e-b7a5-343010cbf5bb None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:10.166567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bbad6-0887-4a7e-b7a5-343010cbf5bb None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:10.185444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:10.205757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:10.206374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:10.206644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:10.206912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:10.217974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 10:41:10.222562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:10.223638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:10.224733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 637/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Aug 12 10:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:10.230543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403cfb3-c5bd-4791-af5e-647509c10dc1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:10.334843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:10.355778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403cfb3-c5bd-4791-af5e-647509c10dc1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:10.393827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:10.394420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:10.394689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:10.394950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:10.395613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:10.395908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:10.397319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:10.406004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 640/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Aug 12 10:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:10.415975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:10.529897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbafc38-ee48-4f4f-aba3-6fc6070678b8 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:10.563853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:10.564466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:10.564712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:10.564974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:10.577111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:10.578573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:41:10.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:10.595021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:41:10.596139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:41:10.596139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:10.596139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:10.599838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 638/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:10.601020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:10.732504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752520b-1e37-4093-8e26-e664826a5e3d None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:10.770745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:10.771332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:10.771599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:10.771862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:10.772526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:10.772821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:10.774224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:10.785804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 641/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:41:10.791810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:10.824688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:10.899909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e55578-3038-4d5e-a431-1e1f82f4d5a4 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:10.939056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:10.939673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:10.939917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:10.940179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:10.948981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:10.951236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 10:41:10.977451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:10.980052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:10.981649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 639/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:10.986248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:10.994431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:11.117672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10754-5baa-4b3a-9678-cfe333cc3a5e None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:11.159615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:11.160217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:11.160535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:11.160803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:11.162091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:11.162431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:11.163886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:11.183953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 642/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Aug 12 10:41:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:11.193002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:11.313989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:11.331913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e453f-6da2-42b2-bf90-b931eae12435 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:11.380068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:11.380690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:11.380960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:11.381229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:11.462884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:11.464244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:11.578411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:11.594969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 640/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Aug 12 10:41:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 10:41:11.601089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:11.692100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:11.736354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7851b7-df33-40e7-bb8a-8b8311ba8e30 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:11.773736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:11.774425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:11.774661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:11.774905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:11.785141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:11.786592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 10:41:11.801389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:11.802442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 10:41:11.806767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 10:41:11.806767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 10:41:11.806767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:11.806767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 643/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:11.808730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:11.909609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9f29c-1139-4296-b199-68d0aa02eca2 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:11.945291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:11.946010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:11.946279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:11.946546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:11.947170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:11.947476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:11.948857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:11.961182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 641/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Aug 12 10:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:11.967640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:12.085888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f736a-66aa-434b-8262-f7ee980824e2 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:12.100303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:12.122081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:12.122693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:12.122960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:12.123221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:12.129913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:12.131366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 10:41:12.143701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:12.144756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: ERROR keystone.server.flask.application Aug 12 10:41:12.146335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 644/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Aug 12 10:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:12.152412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:12.204670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:12.257021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d2401-20f5-41ca-b094-5ea0fe6a3717 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:12.293021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3aed5-bc36-4b79-8381-f3bac41c75ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:12.293606 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3aed5-bc36-4b79-8381-f3bac41c75ff None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:12.293886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3aed5-bc36-4b79-8381-f3bac41c75ff None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:12.294150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3aed5-bc36-4b79-8381-f3bac41c75ff None admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:12.294830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3aed5-bc36-4b79-8381-f3bac41c75ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:12.295129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3aed5-bc36-4b79-8381-f3bac41c75ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:12.296596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3aed5-bc36-4b79-8381-f3bac41c75ff None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:12.309079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 642/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Aug 12 10:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:12.315550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:12.429099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08bec-cb6a-4bc6-aa3c-d56b07904334 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:12.464030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba43844-d44e-4a7a-8a5c-d558be3abbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:12.464650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba43844-d44e-4a7a-8a5c-d558be3abbdd None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:12.464921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba43844-d44e-4a7a-8a5c-d558be3abbdd None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:12.465188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba43844-d44e-4a7a-8a5c-d558be3abbdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:12.465906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba43844-d44e-4a7a-8a5c-d558be3abbdd None admin] RBAC: Adding query filter params (role.id=e65d667683404c438483a1b1252ff2bb, scope.project.id=1e93cd13261f41b4971e1b0a6699a850, user.id=fbb12187c53a40e79fe0227256346aeb) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:12.466212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba43844-d44e-4a7a-8a5c-d558be3abbdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:12.467639 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba43844-d44e-4a7a-8a5c-d558be3abbdd None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:12.473541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 645/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Aug 12 10:41:12 2022] GET /identity/v3/role_assignments?user.id=fbb12187c53a40e79fe0227256346aeb&scope.project.id=1e93cd13261f41b4971e1b0a6699a850&role.id=e65d667683404c438483a1b1252ff2bb => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:14.360918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927d018-1ed3-4641-abda-e67d8b395b98 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:14.361390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927d018-1ed3-4641-abda-e67d8b395b98 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:14.361390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927d018-1ed3-4641-abda-e67d8b395b98 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:14.362778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 643/1288] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:41:14.368641 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10563b3f-93c2-4646-bb65-3e6248e5429e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:14.369121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10563b3f-93c2-4646-bb65-3e6248e5429e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:14.369244 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10563b3f-93c2-4646-bb65-3e6248e5429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:14.387800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-10563b3f-93c2-4646-bb65-3e6248e5429e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:41:14.389031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-10563b3f-93c2-4646-bb65-3e6248e5429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:41:14.419701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:14.460785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 646/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:41:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:41:14.729475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927d018-1ed3-4641-abda-e67d8b395b98 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:14.840553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927d018-1ed3-4641-abda-e67d8b395b98 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:14.851000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:14.876850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:14.877460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:14.877729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:14.877994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] PATH_INFO: `/v3/services/placement` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:14.878557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:14.879921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:14.888938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:14.890096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:14.891192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 644/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:41:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 10:41:14.897035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10563b3f-93c2-4646-bb65-3e6248e5429e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:15.001100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10563b3f-93c2-4646-bb65-3e6248e5429e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:15.036274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbc203-9298-42fb-8a0b-01abe5a88f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:15.037032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbc203-9298-42fb-8a0b-01abe5a88f21 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:15.037297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbc203-9298-42fb-8a0b-01abe5a88f21 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:15.037616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbc203-9298-42fb-8a0b-01abe5a88f21 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:15.038408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbc203-9298-42fb-8a0b-01abe5a88f21 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:15.038719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbc203-9298-42fb-8a0b-01abe5a88f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:15.040781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbc203-9298-42fb-8a0b-01abe5a88f21 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:15.047199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 647/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Aug 12 10:41:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:15.056347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:15.173081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a3b7f-a0c9-4006-8bbb-4fee678fd10b None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:15.207243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29b60f-065e-4420-862b-4fade86412f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:15.207839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29b60f-065e-4420-862b-4fade86412f6 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:15.208110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29b60f-065e-4420-862b-4fade86412f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:15.208414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29b60f-065e-4420-862b-4fade86412f6 None admin] PATH_INFO: `/v3/services` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:15.209031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29b60f-065e-4420-862b-4fade86412f6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:15.209331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29b60f-065e-4420-862b-4fade86412f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:15.210695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29b60f-065e-4420-862b-4fade86412f6 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:15.215359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 645/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Aug 12 10:41:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:17.091884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dddf8-0b8a-4754-9cfd-96ae9d02c505 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:17.092785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dddf8-0b8a-4754-9cfd-96ae9d02c505 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:17.092785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dddf8-0b8a-4754-9cfd-96ae9d02c505 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:17.094987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 648/1293] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:41:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:41:17.101296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719088c9-4b0d-4be2-9a42-5b609f528441 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:17.102086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719088c9-4b0d-4be2-9a42-5b609f528441 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:17.102237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719088c9-4b0d-4be2-9a42-5b609f528441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:17.127264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-719088c9-4b0d-4be2-9a42-5b609f528441 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:41:17.128490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-719088c9-4b0d-4be2-9a42-5b609f528441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:41:17.160594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:17.198552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 646/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:41:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:41:17.463158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dddf8-0b8a-4754-9cfd-96ae9d02c505 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:17.499761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:17.581180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dddf8-0b8a-4754-9cfd-96ae9d02c505 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:17.616374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a695a5b-d357-41bc-afa5-817cb9cb3b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:17.616967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a695a5b-d357-41bc-afa5-817cb9cb3b3f None admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:17.617233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a695a5b-d357-41bc-afa5-817cb9cb3b3f None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:17.617502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a695a5b-d357-41bc-afa5-817cb9cb3b3f None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:17.618158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a695a5b-d357-41bc-afa5-817cb9cb3b3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:17.619561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a695a5b-d357-41bc-afa5-817cb9cb3b3f None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:17.638492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 649/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Aug 12 10:41:17 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:41:19.537850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d636-c463-4469-8d41-d4078834c038 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:19.538724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d636-c463-4469-8d41-d4078834c038 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:19.538724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d636-c463-4469-8d41-d4078834c038 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:19.541340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 647/1296] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:41:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:41:19.546758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e96dc-380a-40cd-8902-3a80b26e4986 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:19.547198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e96dc-380a-40cd-8902-3a80b26e4986 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:19.547288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e96dc-380a-40cd-8902-3a80b26e4986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:19.566843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-e87e96dc-380a-40cd-8902-3a80b26e4986 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:41:19.568015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e87e96dc-380a-40cd-8902-3a80b26e4986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:41:19.646753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 650/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:41:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:41:19.908908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40d636-c463-4469-8d41-d4078834c038 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:20.013215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40d636-c463-4469-8d41-d4078834c038 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:20.051716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:20.052910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:20.053310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:20.054009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] PATH_INFO: `/v3/services/placement` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:20.055372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:20.057148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:20.068922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:20.070169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:20.071276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 648/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:41:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:20.077086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e96dc-380a-40cd-8902-3a80b26e4986 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:20.098638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:20.182622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e96dc-380a-40cd-8902-3a80b26e4986 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:20.218105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:20.218695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:20.218963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:20.219222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:20.219854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:20.220143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:20.221563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:20.226878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 651/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Aug 12 10:41:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:20.233736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:20.267123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:20.334622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3dba3-b83b-4db8-80e5-e478c31fb3bf None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:20.375934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ccc83-7907-4102-9c44-1f4086cdead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:20.376541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ccc83-7907-4102-9c44-1f4086cdead4 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:20.376808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ccc83-7907-4102-9c44-1f4086cdead4 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:20.377069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ccc83-7907-4102-9c44-1f4086cdead4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:20.377755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ccc83-7907-4102-9c44-1f4086cdead4 None admin] RBAC: Adding query filter params (interface=public, service_id=aa658d4fdc5347c8ae32d4b3a85e37d2, region_id=RegionOne) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:20.378053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ccc83-7907-4102-9c44-1f4086cdead4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:20.379411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ccc83-7907-4102-9c44-1f4086cdead4 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:20.384919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 649/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Aug 12 10:41:20 2022] GET /identity/v3/endpoints?service_id=aa658d4fdc5347c8ae32d4b3a85e37d2&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:20.394107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:20.477987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:20.500367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c31c72-9995-41b4-bea8-f0b11570b2b1 None admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:20.537881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc380bab-8a4c-4f5a-9678-563f5a25f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:20.538472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc380bab-8a4c-4f5a-9678-563f5a25f623 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:20.538745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc380bab-8a4c-4f5a-9678-563f5a25f623 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:20.538965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc380bab-8a4c-4f5a-9678-563f5a25f623 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:20.539753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc380bab-8a4c-4f5a-9678-563f5a25f623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:20.541240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc380bab-8a4c-4f5a-9678-563f5a25f623 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:20.548125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 652/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Aug 12 10:41:20 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:41:22.416536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1eb8-42a6-4326-b2e9-8f281cbc76d1 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:22.417370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1eb8-42a6-4326-b2e9-8f281cbc76d1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:22.417370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1eb8-42a6-4326-b2e9-8f281cbc76d1 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:22.419547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 650/1302] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:41:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:41:22.425087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41bfae-ec39-4aec-a5e9-0b8345630955 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:22.425520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41bfae-ec39-4aec-a5e9-0b8345630955 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:22.425610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41bfae-ec39-4aec-a5e9-0b8345630955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:22.444722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7b41bfae-ec39-4aec-a5e9-0b8345630955 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:41:22.445923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7b41bfae-ec39-4aec-a5e9-0b8345630955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:41:22.524638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 653/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:41:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 10:41:22.804366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1eb8-42a6-4326-b2e9-8f281cbc76d1 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:22.918196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:22.925812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1eb8-42a6-4326-b2e9-8f281cbc76d1 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:22.961700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:22.962223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:22.962491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:22.962752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:22.963292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:22.964675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 10:41:22.973783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 10:41:22.974912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application with Lock( Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self._enter() Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application return self.creator() Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application created_value = creator( Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: ERROR keystone.server.flask.application Aug 12 10:41:22.975899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 651/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Aug 12 10:41:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 10:41:22.981653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41bfae-ec39-4aec-a5e9-0b8345630955 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:23.072223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:23.096298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41bfae-ec39-4aec-a5e9-0b8345630955 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:23.141156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a6eb2-a27c-4b7a-a3a5-3d8cc01dce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:23.142026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a6eb2-a27c-4b7a-a3a5-3d8cc01dce52 None admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:23.142459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a6eb2-a27c-4b7a-a3a5-3d8cc01dce52 None admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:23.142876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a6eb2-a27c-4b7a-a3a5-3d8cc01dce52 None admin] PATH_INFO: `/v3/services` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:23.143847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a6eb2-a27c-4b7a-a3a5-3d8cc01dce52 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:41:23.144346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a6eb2-a27c-4b7a-a3a5-3d8cc01dce52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:23.146506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a6eb2-a27c-4b7a-a3a5-3d8cc01dce52 None admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:23.152814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 654/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Aug 12 10:41:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:41:23.158781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:41:23.259495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a759-76c7-435b-9987-ba1e7e640299 None admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:41:23.294317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621758b-6fb3-4cc0-9362-45c29f29ce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:41:23.294899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621758b-6fb3-4cc0-9362-45c29f29ce90 None admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:41:23.295168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621758b-6fb3-4cc0-9362-45c29f29ce90 None admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:41:23.295428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621758b-6fb3-4cc0-9362-45c29f29ce90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:41:23.296064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621758b-6fb3-4cc0-9362-45c29f29ce90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:41:23.297456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621758b-6fb3-4cc0-9362-45c29f29ce90 None admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:41:23.320738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 652/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:41:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:41:33.149677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=30 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:41:33.316631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:26.670831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2a42-2133-427a-b29d-2a7a236940a8 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:26.671399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2a42-2133-427a-b29d-2a7a236940a8 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:26.671399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb2a42-2133-427a-b29d-2a7a236940a8 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:26.676548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 655/1307] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:26.678113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec998-7e9d-4e63-ba83-ebf82751cf3d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:26.678926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec998-7e9d-4e63-ba83-ebf82751cf3d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:26.679784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec998-7e9d-4e63-ba83-ebf82751cf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:26.767275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-6d2ec998-7e9d-4e63-ba83-ebf82751cf3d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:26.768529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6d2ec998-7e9d-4e63-ba83-ebf82751cf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:26.885681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 653/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:27.196193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b113a-4ddf-4b58-87d3-041ffb682169 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:27.196592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b113a-4ddf-4b58-87d3-041ffb682169 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:27.197882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b113a-4ddf-4b58-87d3-041ffb682169 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:27.199654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 656/1309] 10.222.0.23 () {54 vars in 965 bytes} [Fri Aug 12 10:42:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:27.205952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b48b85-76cb-4752-aac4-f526fb4aaee4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:27.206244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b48b85-76cb-4752-aac4-f526fb4aaee4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:27.206520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b48b85-76cb-4752-aac4-f526fb4aaee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:27.291567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-80b48b85-76cb-4752-aac4-f526fb4aaee4 None None] MFA Rules not processed for user `a8f3a56366d348bfb0a5c445905dc46b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:27.292812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-80b48b85-76cb-4752-aac4-f526fb4aaee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:27.314055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:27.378792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 654/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Fri Aug 12 10:42:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:42:27.388185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b113a-4ddf-4b58-87d3-041ffb682169 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:27.389905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-353b113a-4ddf-4b58-87d3-041ffb682169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:27.498773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b113a-4ddf-4b58-87d3-041ffb682169 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:27.583320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9278a8-34a4-4914-88e2-138a5aa4e787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:27.583936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9278a8-34a4-4914-88e2-138a5aa4e787 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:27.584186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9278a8-34a4-4914-88e2-138a5aa4e787 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:27.584509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9278a8-34a4-4914-88e2-138a5aa4e787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:27.586425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-be9278a8-34a4-4914-88e2-138a5aa4e787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:27.649112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:27.711153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9278a8-34a4-4914-88e2-138a5aa4e787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:27.712642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9278a8-34a4-4914-88e2-138a5aa4e787 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:27.732909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 657/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:29.898683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be08855-496b-423b-8cd2-916edb752e7b None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:29.899228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be08855-496b-423b-8cd2-916edb752e7b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:29.899228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be08855-496b-423b-8cd2-916edb752e7b None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:29.906718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd696d1-2b34-41df-9394-aec54b93569e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:29.907672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd696d1-2b34-41df-9394-aec54b93569e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:29.908123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd696d1-2b34-41df-9394-aec54b93569e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:29.909376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 655/1312] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:29 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:29.995047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-cdd696d1-2b34-41df-9394-aec54b93569e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:29.996286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cdd696d1-2b34-41df-9394-aec54b93569e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:30.106020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 658/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:42:30.387191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be08855-496b-423b-8cd2-916edb752e7b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:30.388749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2be08855-496b-423b-8cd2-916edb752e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:30.492236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be08855-496b-423b-8cd2-916edb752e7b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:30.564132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:30.566975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47a782-ef0a-4c8a-ab5b-f26716ebf2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:30.567578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47a782-ef0a-4c8a-ab5b-f26716ebf2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:30.567849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47a782-ef0a-4c8a-ab5b-f26716ebf2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:30.568118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47a782-ef0a-4c8a-ab5b-f26716ebf2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:30.570475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ca47a782-ef0a-4c8a-ab5b-f26716ebf2a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:30.690429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47a782-ef0a-4c8a-ab5b-f26716ebf2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:30.691867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47a782-ef0a-4c8a-ab5b-f26716ebf2a2 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:30.713040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 656/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:32.927487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbfa64-52da-4606-b875-b76f05258694 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:32.928498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbfa64-52da-4606-b875-b76f05258694 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:32.928498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbfa64-52da-4606-b875-b76f05258694 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:32.930330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 659/1315] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:32.936537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8682f-cced-4b31-b72b-eb19b9c36a05 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:32.936955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8682f-cced-4b31-b72b-eb19b9c36a05 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:32.937091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8682f-cced-4b31-b72b-eb19b9c36a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:33.014206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cbe8682f-cced-4b31-b72b-eb19b9c36a05 None None] MFA Rules not processed for user `a4740f00cc764a3ca9358d80c6fb5630`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:33.015419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cbe8682f-cced-4b31-b72b-eb19b9c36a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:33.072572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:33.121156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 657/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:33.411273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbfa64-52da-4606-b875-b76f05258694 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:33.413867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-edfbfa64-52da-4606-b875-b76f05258694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:33.424145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:33.523287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbfa64-52da-4606-b875-b76f05258694 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:33.604314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ede29-e07d-4985-8cae-c57a633e1395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:33.604822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ede29-e07d-4985-8cae-c57a633e1395 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:33.605090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ede29-e07d-4985-8cae-c57a633e1395 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:33.605354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ede29-e07d-4985-8cae-c57a633e1395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:33.606941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f59ede29-e07d-4985-8cae-c57a633e1395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:33.674527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:33.725142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ede29-e07d-4985-8cae-c57a633e1395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:33.726960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ede29-e07d-4985-8cae-c57a633e1395 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:33.747984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 660/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:36.164776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a91a93-28eb-4d31-96f8-4ce0091c6bf4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:36.165368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a91a93-28eb-4d31-96f8-4ce0091c6bf4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:36.165368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a91a93-28eb-4d31-96f8-4ce0091c6bf4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:36.166661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 658/1318] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:36.173184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801d5e3-6f2f-4b16-bf02-1c6df6b01d6f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:36.173869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801d5e3-6f2f-4b16-bf02-1c6df6b01d6f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:36.174061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801d5e3-6f2f-4b16-bf02-1c6df6b01d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:36.262599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6801d5e3-6f2f-4b16-bf02-1c6df6b01d6f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:36.263808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6801d5e3-6f2f-4b16-bf02-1c6df6b01d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:36.368273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 661/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:36.648008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a91a93-28eb-4d31-96f8-4ce0091c6bf4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:36.649589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f3a91a93-28eb-4d31-96f8-4ce0091c6bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:36.765227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a91a93-28eb-4d31-96f8-4ce0091c6bf4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:36.840679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795a21e-b2a4-4fe3-8abe-a86f1b9cbfdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:36.841270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795a21e-b2a4-4fe3-8abe-a86f1b9cbfdd service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:36.841537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795a21e-b2a4-4fe3-8abe-a86f1b9cbfdd service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:36.841798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795a21e-b2a4-4fe3-8abe-a86f1b9cbfdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:36.843364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6795a21e-b2a4-4fe3-8abe-a86f1b9cbfdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:36.851068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:36.965542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795a21e-b2a4-4fe3-8abe-a86f1b9cbfdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:36.967795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795a21e-b2a4-4fe3-8abe-a86f1b9cbfdd service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:36.990502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 659/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:39.584731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a9a05-cf04-4dbf-b043-ff96a70b193e None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:39.586050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a9a05-cf04-4dbf-b043-ff96a70b193e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:39.586050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a9a05-cf04-4dbf-b043-ff96a70b193e None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:39.586740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 662/1321] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:39.592517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844a393-9696-4870-9632-c37d0169a3da None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:39.592940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844a393-9696-4870-9632-c37d0169a3da None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:39.593035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844a393-9696-4870-9632-c37d0169a3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:39.662990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7844a393-9696-4870-9632-c37d0169a3da None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:39.664211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7844a393-9696-4870-9632-c37d0169a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:39.725786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:39.767981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 660/1322] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:40.047500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a9a05-cf04-4dbf-b043-ff96a70b193e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:40.049213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e19a9a05-cf04-4dbf-b043-ff96a70b193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:40.069207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:40.153967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a9a05-cf04-4dbf-b043-ff96a70b193e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:40.229521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee9ecd-dd46-45d9-8388-eaf0578a6854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:40.230035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee9ecd-dd46-45d9-8388-eaf0578a6854 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:40.230280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee9ecd-dd46-45d9-8388-eaf0578a6854 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:40.230584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee9ecd-dd46-45d9-8388-eaf0578a6854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:40.232175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c7ee9ecd-dd46-45d9-8388-eaf0578a6854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:40.307095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:40.354085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee9ecd-dd46-45d9-8388-eaf0578a6854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:40.355535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee9ecd-dd46-45d9-8388-eaf0578a6854 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:40.376207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 663/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:42.339161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45bc0d-c9ee-4280-be6c-4a159ae65bdc None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:42.339599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45bc0d-c9ee-4280-be6c-4a159ae65bdc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:42.339945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45bc0d-c9ee-4280-be6c-4a159ae65bdc None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:42.342003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 661/1324] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:42.349317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49585df2-a90f-4eb4-bb6f-d6f724357577 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:42.349801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49585df2-a90f-4eb4-bb6f-d6f724357577 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:42.349910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49585df2-a90f-4eb4-bb6f-d6f724357577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:42.425914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-49585df2-a90f-4eb4-bb6f-d6f724357577 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:42.427151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-49585df2-a90f-4eb4-bb6f-d6f724357577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:42.525445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 664/1325] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:42.811679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45bc0d-c9ee-4280-be6c-4a159ae65bdc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:42.813265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ef45bc0d-c9ee-4280-be6c-4a159ae65bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:42.931893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45bc0d-c9ee-4280-be6c-4a159ae65bdc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:43.009100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cb20c-ec37-434f-898a-36e6ca5bf5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:43.009711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cb20c-ec37-434f-898a-36e6ca5bf5cb service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:43.009985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cb20c-ec37-434f-898a-36e6ca5bf5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:43.010249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cb20c-ec37-434f-898a-36e6ca5bf5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:43.012180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-398cb20c-ec37-434f-898a-36e6ca5bf5cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:43.029631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:43.131393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cb20c-ec37-434f-898a-36e6ca5bf5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:43.133447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cb20c-ec37-434f-898a-36e6ca5bf5cb service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:43.156137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 662/1326] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:45.105682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09844df9-2a2a-495a-aaef-8b4c722f39ec None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:45.106235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09844df9-2a2a-495a-aaef-8b4c722f39ec None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:45.106235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09844df9-2a2a-495a-aaef-8b4c722f39ec None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:45.107577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 665/1327] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:45.113019 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba470d-b0e0-46f4-8d55-fef3bc55780e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:45.113436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba470d-b0e0-46f4-8d55-fef3bc55780e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:45.113541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba470d-b0e0-46f4-8d55-fef3bc55780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:45.182827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b3ba470d-b0e0-46f4-8d55-fef3bc55780e None None] MFA Rules not processed for user `a4740f00cc764a3ca9358d80c6fb5630`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:45.183963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b3ba470d-b0e0-46f4-8d55-fef3bc55780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:45.252180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:45.284597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 663/1328] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:45.570201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09844df9-2a2a-495a-aaef-8b4c722f39ec None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:45.571786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-09844df9-2a2a-495a-aaef-8b4c722f39ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:45.604601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:45.684692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09844df9-2a2a-495a-aaef-8b4c722f39ec None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:45.774791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02feb9-c0f7-453f-9307-c0211d92179e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:45.775421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02feb9-c0f7-453f-9307-c0211d92179e service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:45.775686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02feb9-c0f7-453f-9307-c0211d92179e service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:45.775948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02feb9-c0f7-453f-9307-c0211d92179e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:45.777605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6a02feb9-c0f7-453f-9307-c0211d92179e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:45.855901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:45.887773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02feb9-c0f7-453f-9307-c0211d92179e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:45.889280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02feb9-c0f7-453f-9307-c0211d92179e service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:45.907394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 666/1329] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:48.035578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66da418-071b-4543-bf1e-2188e58c7e89 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:48.036480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66da418-071b-4543-bf1e-2188e58c7e89 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:48.036480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66da418-071b-4543-bf1e-2188e58c7e89 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:48.038738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 664/1330] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:48.047338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab62bb8-e952-4b49-947f-33b7beedc52e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:48.047835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab62bb8-e952-4b49-947f-33b7beedc52e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:48.048089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab62bb8-e952-4b49-947f-33b7beedc52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:48.117563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6ab62bb8-e952-4b49-947f-33b7beedc52e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:48.118765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6ab62bb8-e952-4b49-947f-33b7beedc52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:48.226769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 667/1331] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:48.503894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66da418-071b-4543-bf1e-2188e58c7e89 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:48.505562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d66da418-071b-4543-bf1e-2188e58c7e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:48.613145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66da418-071b-4543-bf1e-2188e58c7e89 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:48.695234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b8392-97e3-444a-acaf-e88f36cdde1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:48.696193 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b8392-97e3-444a-acaf-e88f36cdde1a service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:48.696724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b8392-97e3-444a-acaf-e88f36cdde1a service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:48.697170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b8392-97e3-444a-acaf-e88f36cdde1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:48.699793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2f7b8392-97e3-444a-acaf-e88f36cdde1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:48.733690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:48.823076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b8392-97e3-444a-acaf-e88f36cdde1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:48.825323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b8392-97e3-444a-acaf-e88f36cdde1a service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:48.849895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 665/1332] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:50.794766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb82898-08d1-4705-b5bd-77a304a5534b None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:50.795352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb82898-08d1-4705-b5bd-77a304a5534b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:50.795352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb82898-08d1-4705-b5bd-77a304a5534b None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:50.796697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 668/1333] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:50.802975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983859ff-f1cb-4a8a-97e1-c39be1ac2b9d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:50.804512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983859ff-f1cb-4a8a-97e1-c39be1ac2b9d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:50.804512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983859ff-f1cb-4a8a-97e1-c39be1ac2b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:51.011035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-983859ff-f1cb-4a8a-97e1-c39be1ac2b9d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:51.012616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-983859ff-f1cb-4a8a-97e1-c39be1ac2b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:51.108928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:51.135188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 666/1334] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:51.443366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492ffb9-38c5-4af3-84e0-5b1c4da9e77c None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:51.443693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492ffb9-38c5-4af3-84e0-5b1c4da9e77c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:51.443985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492ffb9-38c5-4af3-84e0-5b1c4da9e77c None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:51.446345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 669/1335] 10.222.0.23 () {54 vars in 965 bytes} [Fri Aug 12 10:42:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:51.454073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cd6cc-b9a8-4782-aeaf-97bb1a7f70bb None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:51.454349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cd6cc-b9a8-4782-aeaf-97bb1a7f70bb None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:51.454609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cd6cc-b9a8-4782-aeaf-97bb1a7f70bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:51.535591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7c2cd6cc-b9a8-4782-aeaf-97bb1a7f70bb None None] MFA Rules not processed for user `a8f3a56366d348bfb0a5c445905dc46b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:51.536829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7c2cd6cc-b9a8-4782-aeaf-97bb1a7f70bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:51.624140 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 667/1336] 10.222.0.23 () {58 vars in 1062 bytes} [Fri Aug 12 10:42:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:51.633899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492ffb9-38c5-4af3-84e0-5b1c4da9e77c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:51.635396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4492ffb9-38c5-4af3-84e0-5b1c4da9e77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:51.675704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:51.755695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492ffb9-38c5-4af3-84e0-5b1c4da9e77c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:51.835783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6475a7-9698-47f4-bb15-4bca982c9325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:51.836563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6475a7-9698-47f4-bb15-4bca982c9325 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:51.836666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6475a7-9698-47f4-bb15-4bca982c9325 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:51.836911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6475a7-9698-47f4-bb15-4bca982c9325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:51.838479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2a6475a7-9698-47f4-bb15-4bca982c9325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:51.924449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:51.951709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6475a7-9698-47f4-bb15-4bca982c9325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:51.953111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6475a7-9698-47f4-bb15-4bca982c9325 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:51.970900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 670/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:54.713852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6477-8f53-4eff-839a-bc50613706e0 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:54.714413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6477-8f53-4eff-839a-bc50613706e0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:54.714413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6477-8f53-4eff-839a-bc50613706e0 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:54.715749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 668/1338] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:54.722304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad06c-fb16-4e3d-87af-ed150e277d60 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:54.723003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad06c-fb16-4e3d-87af-ed150e277d60 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:54.723147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad06c-fb16-4e3d-87af-ed150e277d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:54.798901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6b5ad06c-fb16-4e3d-87af-ed150e277d60 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:54.800073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6b5ad06c-fb16-4e3d-87af-ed150e277d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:54.906080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 671/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:55.189066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a6477-8f53-4eff-839a-bc50613706e0 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:55.191382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6e0a6477-8f53-4eff-839a-bc50613706e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:55.281286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:55.309834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a6477-8f53-4eff-839a-bc50613706e0 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:42:55.390224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77351f8-c281-439c-91e2-7aa7685abbb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:42:55.390822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77351f8-c281-439c-91e2-7aa7685abbb8 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:55.391096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77351f8-c281-439c-91e2-7aa7685abbb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:55.391359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77351f8-c281-439c-91e2-7aa7685abbb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:55.392965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c77351f8-c281-439c-91e2-7aa7685abbb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:55.524078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77351f8-c281-439c-91e2-7aa7685abbb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:42:55.525493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77351f8-c281-439c-91e2-7aa7685abbb8 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:42:55.544203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 669/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:42:59.496606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887511d8-2f69-48ce-932c-fa7c04ce7721 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:59.497081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887511d8-2f69-48ce-932c-fa7c04ce7721 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:59.497173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887511d8-2f69-48ce-932c-fa7c04ce7721 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:59.498476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 672/1341] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:42:59.505575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156ac66-6f22-4554-81b2-5b7152973962 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:42:59.506071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156ac66-6f22-4554-81b2-5b7152973962 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:42:59.506166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156ac66-6f22-4554-81b2-5b7152973962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:42:59.512687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:42:59.577690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b156ac66-6f22-4554-81b2-5b7152973962 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:42:59.578874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b156ac66-6f22-4554-81b2-5b7152973962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:42:59.678171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 670/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:42:59.954002 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887511d8-2f69-48ce-932c-fa7c04ce7721 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:42:59.955531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-887511d8-2f69-48ce-932c-fa7c04ce7721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:00.004138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:00.063851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887511d8-2f69-48ce-932c-fa7c04ce7721 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:00.139226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40735ec0-de66-4ae3-8750-d78a7e0ad9b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:00.139838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735ec0-de66-4ae3-8750-d78a7e0ad9b7 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:00.140120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735ec0-de66-4ae3-8750-d78a7e0ad9b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:00.140416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735ec0-de66-4ae3-8750-d78a7e0ad9b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:00.141995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-40735ec0-de66-4ae3-8750-d78a7e0ad9b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:00.239483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:00.257154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40735ec0-de66-4ae3-8750-d78a7e0ad9b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:00.258599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40735ec0-de66-4ae3-8750-d78a7e0ad9b7 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:00.277396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 673/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:02.799264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44556b0-8be4-4f80-8d5c-9866ff83bac4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:02.799848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44556b0-8be4-4f80-8d5c-9866ff83bac4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:02.799848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44556b0-8be4-4f80-8d5c-9866ff83bac4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:02.801230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 671/1344] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:02.807217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed5ee-8bae-44bd-9e08-6810dadc32cc None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:02.807781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed5ee-8bae-44bd-9e08-6810dadc32cc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:02.807781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed5ee-8bae-44bd-9e08-6810dadc32cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:02.880688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-316ed5ee-8bae-44bd-9e08-6810dadc32cc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:02.882039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-316ed5ee-8bae-44bd-9e08-6810dadc32cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:02.981092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 674/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:03.270144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44556b0-8be4-4f80-8d5c-9866ff83bac4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:03.271699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d44556b0-8be4-4f80-8d5c-9866ff83bac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:03.364151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:03.383463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44556b0-8be4-4f80-8d5c-9866ff83bac4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:03.460369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5f2ff-f963-4c2e-ae32-a5c43c7da1c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:03.460985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f2ff-f963-4c2e-ae32-a5c43c7da1c5 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:03.461264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f2ff-f963-4c2e-ae32-a5c43c7da1c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:03.461532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f2ff-f963-4c2e-ae32-a5c43c7da1c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:03.463143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-58d5f2ff-f963-4c2e-ae32-a5c43c7da1c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:03.589842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5f2ff-f963-4c2e-ae32-a5c43c7da1c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:03.591201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5f2ff-f963-4c2e-ae32-a5c43c7da1c5 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:03.609473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 672/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:07.799916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856392c9-e9d5-42d0-b1f9-2e6c052c1b8c None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:07.800567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856392c9-e9d5-42d0-b1f9-2e6c052c1b8c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:07.800567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856392c9-e9d5-42d0-b1f9-2e6c052c1b8c None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:07.801846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 675/1347] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:07.807406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992b88-b422-4cf4-a961-0c00aec9d2d7 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:07.807882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992b88-b422-4cf4-a961-0c00aec9d2d7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:07.807882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992b88-b422-4cf4-a961-0c00aec9d2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:07.825032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:07.880045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-6c992b88-b422-4cf4-a961-0c00aec9d2d7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:07.881322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6c992b88-b422-4cf4-a961-0c00aec9d2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:07.984621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 673/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:08.271515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856392c9-e9d5-42d0-b1f9-2e6c052c1b8c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:08.273162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-856392c9-e9d5-42d0-b1f9-2e6c052c1b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:08.352963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:08.406822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856392c9-e9d5-42d0-b1f9-2e6c052c1b8c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:08.484500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a52ce-267c-4fd7-b144-edcb23bb1514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:08.485126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a52ce-267c-4fd7-b144-edcb23bb1514 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:08.485395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a52ce-267c-4fd7-b144-edcb23bb1514 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:08.485660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a52ce-267c-4fd7-b144-edcb23bb1514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:08.487297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b99a52ce-267c-4fd7-b144-edcb23bb1514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:08.599752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:08.604849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a52ce-267c-4fd7-b144-edcb23bb1514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:08.606427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a52ce-267c-4fd7-b144-edcb23bb1514 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:08.625313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 676/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:13.638026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c96c4-2722-4a10-a5a0-fedac1e4a2b7 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:13.638500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c96c4-2722-4a10-a5a0-fedac1e4a2b7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:13.639718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c96c4-2722-4a10-a5a0-fedac1e4a2b7 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:13.649161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 674/1350] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:43:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:13.657285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b57356-5753-4601-b9c3-f7017b7b6714 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:13.658319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b57356-5753-4601-b9c3-f7017b7b6714 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:13.658785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b57356-5753-4601-b9c3-f7017b7b6714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:13.773041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a7b57356-5753-4601-b9c3-f7017b7b6714 None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:13.774354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a7b57356-5753-4601-b9c3-f7017b7b6714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:13.905224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 677/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:43:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:17.982717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=59 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:18.027344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe789-f669-49c8-983e-81fdb28749f9 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:18.027618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe789-f669-49c8-983e-81fdb28749f9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:18.028095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe789-f669-49c8-983e-81fdb28749f9 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:18.029593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 675/1352] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:18.034853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c423b5-fbe2-4b88-8001-280f142b516f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:18.035346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c423b5-fbe2-4b88-8001-280f142b516f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:18.035604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c423b5-fbe2-4b88-8001-280f142b516f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:18.091026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:18.116525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-65c423b5-fbe2-4b88-8001-280f142b516f None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:18.117860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-65c423b5-fbe2-4b88-8001-280f142b516f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:18.235344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 678/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:19.454984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eadff-88a7-41b9-9014-e830ccbc6539 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:19.455593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eadff-88a7-41b9-9014-e830ccbc6539 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:19.455593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eadff-88a7-41b9-9014-e830ccbc6539 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:19.457033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 676/1354] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:19.462127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3ff04-b2c5-41ea-8901-11d6aae084ed None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:19.462540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3ff04-b2c5-41ea-8901-11d6aae084ed None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:19.462791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3ff04-b2c5-41ea-8901-11d6aae084ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:19.542907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-c6f3ff04-b2c5-41ea-8901-11d6aae084ed None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:19.544123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c6f3ff04-b2c5-41ea-8901-11d6aae084ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:19.590073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:19.657497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 679/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:43:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:43:22.958074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db27463-fd16-4b7a-9dd0-0300d3374278 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:22.958628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db27463-fd16-4b7a-9dd0-0300d3374278 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:22.958628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db27463-fd16-4b7a-9dd0-0300d3374278 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:22.960601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 677/1356] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:22.965786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7948f-7947-47d8-af84-be27ce5c0cf7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:22.966496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7948f-7947-47d8-af84-be27ce5c0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:22.966899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7948f-7947-47d8-af84-be27ce5c0cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:23.039347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-c3d7948f-7947-47d8-af84-be27ce5c0cf7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:23.040571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c3d7948f-7947-47d8-af84-be27ce5c0cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:23.163640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 680/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:23.459212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73df8-1f39-4ce4-8004-a060a97bbe59 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:23.459977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73df8-1f39-4ce4-8004-a060a97bbe59 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:23.461041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73df8-1f39-4ce4-8004-a060a97bbe59 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:23.462894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 678/1358] 10.222.0.23 () {54 vars in 962 bytes} [Fri Aug 12 10:43:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:23.471345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f184f80-997f-46b5-8208-894a9346a450 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:23.472573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f184f80-997f-46b5-8208-894a9346a450 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:23.472937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f184f80-997f-46b5-8208-894a9346a450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:23.491106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:23.553664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2f184f80-997f-46b5-8208-894a9346a450 None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:23.554896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2f184f80-997f-46b5-8208-894a9346a450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:23.691240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 681/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Aug 12 10:43:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:43:23.699823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f73df8-1f39-4ce4-8004-a060a97bbe59 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:23.701404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-01f73df8-1f39-4ce4-8004-a060a97bbe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:23.866862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f73df8-1f39-4ce4-8004-a060a97bbe59 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:23.946623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:23.989969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a58c3-855f-4041-a9ed-8473061e6742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:23.990750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a58c3-855f-4041-a9ed-8473061e6742 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:23.991142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a58c3-855f-4041-a9ed-8473061e6742 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:23.991441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a58c3-855f-4041-a9ed-8473061e6742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:23.993119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-040a58c3-855f-4041-a9ed-8473061e6742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:24.090555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a58c3-855f-4041-a9ed-8473061e6742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:24.092165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a58c3-855f-4041-a9ed-8473061e6742 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:24.124457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 679/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:26.215567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a581ea8-629f-47cd-976f-9c11b9b67797 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:26.216135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a581ea8-629f-47cd-976f-9c11b9b67797 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:26.216135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a581ea8-629f-47cd-976f-9c11b9b67797 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:26.217699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 682/1361] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:26.222990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93b718-2935-48de-82ac-3aff347f45b9 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:26.223469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93b718-2935-48de-82ac-3aff347f45b9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:26.223469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93b718-2935-48de-82ac-3aff347f45b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:26.293766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0e93b718-2935-48de-82ac-3aff347f45b9 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:26.295288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0e93b718-2935-48de-82ac-3aff347f45b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:26.339658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:26.403796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 680/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:26.687629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a581ea8-629f-47cd-976f-9c11b9b67797 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:26.689210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0a581ea8-629f-47cd-976f-9c11b9b67797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:26.764734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:26.850963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a581ea8-629f-47cd-976f-9c11b9b67797 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:26.968287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6725395-c971-410c-bfc4-e921cc384ea0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:26.968911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725395-c971-410c-bfc4-e921cc384ea0 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:26.969157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725395-c971-410c-bfc4-e921cc384ea0 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:26.969458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725395-c971-410c-bfc4-e921cc384ea0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:26.971051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c6725395-c971-410c-bfc4-e921cc384ea0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:26.994527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:27.070938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6725395-c971-410c-bfc4-e921cc384ea0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:27.072583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6725395-c971-410c-bfc4-e921cc384ea0 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:27.093570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 683/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:29.311520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82da4f-eb8b-424e-98b5-fb65fe4483ed None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:29.312096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82da4f-eb8b-424e-98b5-fb65fe4483ed None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:29.312096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82da4f-eb8b-424e-98b5-fb65fe4483ed None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:29.313436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 681/1364] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:29.318916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb05dd2-bfdd-4169-a063-acc1b654c007 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:29.319397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb05dd2-bfdd-4169-a063-acc1b654c007 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:29.319686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb05dd2-bfdd-4169-a063-acc1b654c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:29.399201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-bbb05dd2-bfdd-4169-a063-acc1b654c007 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:29.400471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bbb05dd2-bfdd-4169-a063-acc1b654c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:29.505092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:29.508892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 684/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:29.796218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7fd39-9125-421e-b6b6-881e3560b454 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:29.796740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7fd39-9125-421e-b6b6-881e3560b454 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:29.797173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7fd39-9125-421e-b6b6-881e3560b454 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:29.798974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 682/1366] 10.222.0.23 () {54 vars in 962 bytes} [Fri Aug 12 10:43:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:29.804760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd1b3d-b1fa-413e-80f2-13f211229b0f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:29.805164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd1b3d-b1fa-413e-80f2-13f211229b0f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:29.805533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd1b3d-b1fa-413e-80f2-13f211229b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:29.885967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-cbfd1b3d-b1fa-413e-80f2-13f211229b0f None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:29.888226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cbfd1b3d-b1fa-413e-80f2-13f211229b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:30.019469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 685/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Aug 12 10:43:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:30.027850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7fd39-9125-421e-b6b6-881e3560b454 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:30.029508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-68f7fd39-9125-421e-b6b6-881e3560b454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:30.189440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7fd39-9125-421e-b6b6-881e3560b454 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:30.210707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:30.308795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06546b4-e3f2-4fa4-9343-cc8cf3fac1c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:30.309596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06546b4-e3f2-4fa4-9343-cc8cf3fac1c3 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:30.309973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06546b4-e3f2-4fa4-9343-cc8cf3fac1c3 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:30.310337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06546b4-e3f2-4fa4-9343-cc8cf3fac1c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:30.312094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f06546b4-e3f2-4fa4-9343-cc8cf3fac1c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:30.406798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06546b4-e3f2-4fa4-9343-cc8cf3fac1c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:30.408448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06546b4-e3f2-4fa4-9343-cc8cf3fac1c3 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:30.428706 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 683/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:30.982460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff981dff-a0a0-436e-9f6d-905a413b6902 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:30.983071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff981dff-a0a0-436e-9f6d-905a413b6902 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:30.983071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff981dff-a0a0-436e-9f6d-905a413b6902 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:30.984510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 686/1369] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:30.989932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfa2ac-016e-43ee-898a-f907d9183fe3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:30.990217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfa2ac-016e-43ee-898a-f907d9183fe3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:30.990611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfa2ac-016e-43ee-898a-f907d9183fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:31.026624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:31.068148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f0bfa2ac-016e-43ee-898a-f907d9183fe3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:31.069419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f0bfa2ac-016e-43ee-898a-f907d9183fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:31.172299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 684/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:43:31.745229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0b568-db4b-4f92-a8db-ffb9b7f97857 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:31.745229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0b568-db4b-4f92-a8db-ffb9b7f97857 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:31.745229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0b568-db4b-4f92-a8db-ffb9b7f97857 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:31.748298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 687/1371] 10.222.0.23 () {54 vars in 964 bytes} [Fri Aug 12 10:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:31.751891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377225a8-0e1a-4b0e-af7e-f143e600de3a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:31.752379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377225a8-0e1a-4b0e-af7e-f143e600de3a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:31.752650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377225a8-0e1a-4b0e-af7e-f143e600de3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:31.833748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-377225a8-0e1a-4b0e-af7e-f143e600de3a None None] MFA Rules not processed for user `76f4fb448f1443b7a77fd8e1954defe5`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:31.835435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-377225a8-0e1a-4b0e-af7e-f143e600de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:31.886692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:31.922501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 685/1372] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Aug 12 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:31.930651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0b568-db4b-4f92-a8db-ffb9b7f97857 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:31.932493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-20e0b568-db4b-4f92-a8db-ffb9b7f97857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:31.986176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:32.043738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0b568-db4b-4f92-a8db-ffb9b7f97857 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:32.127348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24799016-146c-4e7a-a317-5de97c02b4f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:32.127959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24799016-146c-4e7a-a317-5de97c02b4f0 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:32.128210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24799016-146c-4e7a-a317-5de97c02b4f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:32.128525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24799016-146c-4e7a-a317-5de97c02b4f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:32.130273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-24799016-146c-4e7a-a317-5de97c02b4f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:32.240381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:32.256350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24799016-146c-4e7a-a317-5de97c02b4f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:32.258378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24799016-146c-4e7a-a317-5de97c02b4f0 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:32.281055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 688/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:32.400326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbadaf6-af7b-4b9d-91c5-d5acb216b773 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:32.400735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbadaf6-af7b-4b9d-91c5-d5acb216b773 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:32.401036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbadaf6-af7b-4b9d-91c5-d5acb216b773 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:32.402559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 686/1374] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:32.408031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a27ea-5f60-4dd8-9c40-9342edad82e0 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:32.408211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a27ea-5f60-4dd8-9c40-9342edad82e0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:32.408816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a27ea-5f60-4dd8-9c40-9342edad82e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:32.477291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0f1a27ea-5f60-4dd8-9c40-9342edad82e0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:32.478547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0f1a27ea-5f60-4dd8-9c40-9342edad82e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:32.579561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 689/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:32.868361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbadaf6-af7b-4b9d-91c5-d5acb216b773 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:32.869959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-edbadaf6-af7b-4b9d-91c5-d5acb216b773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:33.021457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbadaf6-af7b-4b9d-91c5-d5acb216b773 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:33.073746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:33.135956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c821e4a-6f84-4dfd-be5f-d8cae1971a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:33.136510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821e4a-6f84-4dfd-be5f-d8cae1971a73 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:33.136783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821e4a-6f84-4dfd-be5f-d8cae1971a73 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:33.137374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821e4a-6f84-4dfd-be5f-d8cae1971a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:33.138966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4c821e4a-6f84-4dfd-be5f-d8cae1971a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:33.230444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821e4a-6f84-4dfd-be5f-d8cae1971a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:33.232029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c821e4a-6f84-4dfd-be5f-d8cae1971a73 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:33.250449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 687/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:35.201874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc948b-5c35-4019-867c-07c79d1d38b9 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:35.202379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc948b-5c35-4019-867c-07c79d1d38b9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:35.202379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc948b-5c35-4019-867c-07c79d1d38b9 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:35.203826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 690/1377] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:35.209419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17806d2-e5e3-4d43-9a64-9d10b5117772 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:35.209857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17806d2-e5e3-4d43-9a64-9d10b5117772 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:35.209967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17806d2-e5e3-4d43-9a64-9d10b5117772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:35.293570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f17806d2-e5e3-4d43-9a64-9d10b5117772 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:35.294684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f17806d2-e5e3-4d43-9a64-9d10b5117772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:35.301632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:35.395249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 688/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:43:35.688350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc948b-5c35-4019-867c-07c79d1d38b9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:35.689863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-59fc948b-5c35-4019-867c-07c79d1d38b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:35.765392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:35.868633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc948b-5c35-4019-867c-07c79d1d38b9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:35.992254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260744d0-ac26-48ef-9af1-16d22eba5707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:35.992883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260744d0-ac26-48ef-9af1-16d22eba5707 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:35.993156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260744d0-ac26-48ef-9af1-16d22eba5707 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:35.993425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260744d0-ac26-48ef-9af1-16d22eba5707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:35.995023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-260744d0-ac26-48ef-9af1-16d22eba5707 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:36.013794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:36.088868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260744d0-ac26-48ef-9af1-16d22eba5707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:36.090476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260744d0-ac26-48ef-9af1-16d22eba5707 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:36.109674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 691/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:38.040945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51bff-76dd-403a-bcc9-a7c6847e6581 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:38.041486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51bff-76dd-403a-bcc9-a7c6847e6581 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:38.041486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51bff-76dd-403a-bcc9-a7c6847e6581 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:38.043015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 689/1380] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:38.048389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da0324-854e-43d4-8c74-50a557a2d4f9 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:38.048842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da0324-854e-43d4-8c74-50a557a2d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:38.049119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da0324-854e-43d4-8c74-50a557a2d4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:38.118744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-43da0324-854e-43d4-8c74-50a557a2d4f9 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:38.119947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-43da0324-854e-43d4-8c74-50a557a2d4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:38.225914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:38.234559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 692/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:38.519113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b51bff-76dd-403a-bcc9-a7c6847e6581 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:38.520850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b7b51bff-76dd-403a-bcc9-a7c6847e6581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:38.677399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:38.690912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b51bff-76dd-403a-bcc9-a7c6847e6581 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:38.817031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815b926-bc98-45d8-bcc1-012ea19a327e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:38.817833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815b926-bc98-45d8-bcc1-012ea19a327e service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:38.818216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815b926-bc98-45d8-bcc1-012ea19a327e service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:38.818613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815b926-bc98-45d8-bcc1-012ea19a327e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:38.820413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0815b926-bc98-45d8-bcc1-012ea19a327e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:38.913069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815b926-bc98-45d8-bcc1-012ea19a327e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:38.914739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815b926-bc98-45d8-bcc1-012ea19a327e service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:38.925828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:38.935574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 690/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:41.209837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922f452-306b-4b05-a30d-cd8ab31861d7 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:41.210554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922f452-306b-4b05-a30d-cd8ab31861d7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:41.210554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922f452-306b-4b05-a30d-cd8ab31861d7 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:41.211824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 693/1383] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:41.218575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e26efb-9e74-47ed-be54-4c1daef1e26d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:41.219070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e26efb-9e74-47ed-be54-4c1daef1e26d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:41.219298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e26efb-9e74-47ed-be54-4c1daef1e26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:41.293170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-52e26efb-9e74-47ed-be54-4c1daef1e26d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:41.294485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-52e26efb-9e74-47ed-be54-4c1daef1e26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:41.411212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 691/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:41.801545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922f452-306b-4b05-a30d-cd8ab31861d7 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:41.803245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d922f452-306b-4b05-a30d-cd8ab31861d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:41.959564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922f452-306b-4b05-a30d-cd8ab31861d7 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:42.030239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:42.080935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f781423-a3be-4ed9-84fd-761d32a5b28f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:42.081445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f781423-a3be-4ed9-84fd-761d32a5b28f service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:42.081880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f781423-a3be-4ed9-84fd-761d32a5b28f service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:42.082035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f781423-a3be-4ed9-84fd-761d32a5b28f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:42.083590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3f781423-a3be-4ed9-84fd-761d32a5b28f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:42.176725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f781423-a3be-4ed9-84fd-761d32a5b28f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:42.178385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f781423-a3be-4ed9-84fd-761d32a5b28f service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:42.196798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 694/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:44.969188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1d0f-41ee-42ae-95d8-0a13ff2b9db5 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:44.969788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1d0f-41ee-42ae-95d8-0a13ff2b9db5 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:44.969788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1d0f-41ee-42ae-95d8-0a13ff2b9db5 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:44.971089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 692/1386] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:44.977485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130bf47-bbc6-42f6-801f-b286135e2a84 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:44.977933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130bf47-bbc6-42f6-801f-b286135e2a84 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:44.978044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130bf47-bbc6-42f6-801f-b286135e2a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:45.058821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-c130bf47-bbc6-42f6-801f-b286135e2a84 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:45.060043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c130bf47-bbc6-42f6-801f-b286135e2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:45.094357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:45.161556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 695/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:45.446380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1d0f-41ee-42ae-95d8-0a13ff2b9db5 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:45.447887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-633b1d0f-41ee-42ae-95d8-0a13ff2b9db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:45.529669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:45.601513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1d0f-41ee-42ae-95d8-0a13ff2b9db5 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:45.717548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eef76-beb1-40bc-8c92-1c05f740a85f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:45.718158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eef76-beb1-40bc-8c92-1c05f740a85f service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:45.718433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eef76-beb1-40bc-8c92-1c05f740a85f service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:45.718703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eef76-beb1-40bc-8c92-1c05f740a85f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:45.720384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-134eef76-beb1-40bc-8c92-1c05f740a85f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:45.759853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:45.814062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eef76-beb1-40bc-8c92-1c05f740a85f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:45.815666 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eef76-beb1-40bc-8c92-1c05f740a85f service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:45.834191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 693/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:47.433518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9aba5-f1f0-4244-9ae3-e189a68f0aea None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:47.434091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9aba5-f1f0-4244-9ae3-e189a68f0aea None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:47.434091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9aba5-f1f0-4244-9ae3-e189a68f0aea None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:47.435424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 696/1389] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:47.440863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a04d-0ea5-427d-b67e-650540b7e90f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:47.441278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a04d-0ea5-427d-b67e-650540b7e90f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:47.441551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a04d-0ea5-427d-b67e-650540b7e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:47.511739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-6294a04d-0ea5-427d-b67e-650540b7e90f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:47.513297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6294a04d-0ea5-427d-b67e-650540b7e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:47.615387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 694/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:47.737737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8afdde-be8b-41cf-8450-b7e4ec129d73 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:47.738013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8afdde-be8b-41cf-8450-b7e4ec129d73 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:47.738279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8afdde-be8b-41cf-8450-b7e4ec129d73 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:47.739658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 697/1391] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:47.745221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ecf7-10bb-438d-8bef-11cec1ca1c72 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:47.745500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ecf7-10bb-438d-8bef-11cec1ca1c72 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:47.745763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ecf7-10bb-438d-8bef-11cec1ca1c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:47.762842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:47.814509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-eb02ecf7-10bb-438d-8bef-11cec1ca1c72 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:47.815717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-eb02ecf7-10bb-438d-8bef-11cec1ca1c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:47.918350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 695/1392] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:47.944550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e3f57-1ab9-494f-9d56-3bb363e09d57 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:47.944827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e3f57-1ab9-494f-9d56-3bb363e09d57 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:47.945079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e3f57-1ab9-494f-9d56-3bb363e09d57 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:47.946406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 698/1393] 10.222.0.23 () {54 vars in 967 bytes} [Fri Aug 12 10:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:47.951459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:47.951747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:47.952015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:48.033139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] MFA Rules not processed for user `8571f3d80b0749729012353fcbbd9e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:48.034363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:48.073253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:48.116900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 696/1394] 10.222.0.23 () {58 vars in 1064 bytes} [Fri Aug 12 10:43:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:48.125240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e3f57-1ab9-494f-9d56-3bb363e09d57 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:48.126777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-292e3f57-1ab9-494f-9d56-3bb363e09d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:48.212458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:48.213782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:48.244923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e3f57-1ab9-494f-9d56-3bb363e09d57 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:48.302139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:48.317894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b6cf5-1c49-4255-9fb3-2e476e37f856 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8571f3d80b0749729012353fcbbd9e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:48.318566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b6cf5-1c49-4255-9fb3-2e476e37f856 service glance] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:48.318839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b6cf5-1c49-4255-9fb3-2e476e37f856 service glance] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:48.319102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b6cf5-1c49-4255-9fb3-2e476e37f856 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:48.320726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-043b6cf5-1c49-4255-9fb3-2e476e37f856 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:48.338333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c476e-8641-4dfa-84aa-d3ae5ecdcb9a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:48.406405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:48.433066 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b6cf5-1c49-4255-9fb3-2e476e37f856 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:48.434514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b6cf5-1c49-4255-9fb3-2e476e37f856 service glance] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:48.452997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 699/1395] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:48.460279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444a8e6-cb78-4979-87db-0e83007c09bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:48.460890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444a8e6-cb78-4979-87db-0e83007c09bb service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:48.461164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444a8e6-cb78-4979-87db-0e83007c09bb service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:48.461431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444a8e6-cb78-4979-87db-0e83007c09bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:48.463017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d444a8e6-cb78-4979-87db-0e83007c09bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:48.555920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444a8e6-cb78-4979-87db-0e83007c09bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:48.557555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444a8e6-cb78-4979-87db-0e83007c09bb service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:48.576065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 697/1396] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:49.195003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e87e7-8eed-4e8a-855a-4982fd98ba7a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:49.195399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e87e7-8eed-4e8a-855a-4982fd98ba7a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:49.195725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e87e7-8eed-4e8a-855a-4982fd98ba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:49.262928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-013e87e7-8eed-4e8a-855a-4982fd98ba7a None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:49.264212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-013e87e7-8eed-4e8a-855a-4982fd98ba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:49.309330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:49.357184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 700/1397] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:49.406460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0c383-96d5-42dd-9c77-ad555998a477 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:49.406742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0c383-96d5-42dd-9c77-ad555998a477 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:49.407011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0c383-96d5-42dd-9c77-ad555998a477 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:49.408405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 698/1398] 10.222.0.23 () {54 vars in 943 bytes} [Fri Aug 12 10:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:49.414622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a202-0da6-4021-bd62-b5951249e6f3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:49.414953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a202-0da6-4021-bd62-b5951249e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:49.415227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a202-0da6-4021-bd62-b5951249e6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:49.495011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-bb22a202-0da6-4021-bd62-b5951249e6f3 None None] MFA Rules not processed for user `01afe498af8c42e8bca4e993897ad1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:49.496194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bb22a202-0da6-4021-bd62-b5951249e6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:49.602945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 701/1399] 10.222.0.23 () {58 vars in 1040 bytes} [Fri Aug 12 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:49.612381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0c383-96d5-42dd-9c77-ad555998a477 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:49.613973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5ff0c383-96d5-42dd-9c77-ad555998a477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:49.693337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:49.758607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0c383-96d5-42dd-9c77-ad555998a477 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:49.860556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f472e2-b4ee-41ec-b063-89686b8c8cb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:49.861057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f472e2-b4ee-41ec-b063-89686b8c8cb5 service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:49.861323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f472e2-b4ee-41ec-b063-89686b8c8cb5 service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:49.861584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f472e2-b4ee-41ec-b063-89686b8c8cb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:49.863161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b3f472e2-b4ee-41ec-b063-89686b8c8cb5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:49.923416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:49.946333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f472e2-b4ee-41ec-b063-89686b8c8cb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:49.947922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f472e2-b4ee-41ec-b063-89686b8c8cb5 service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:49.956287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 699/1400] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:43:50.507605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f957d-74ed-4ef0-a09c-1c6c53000e5e None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:50.507605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f957d-74ed-4ef0-a09c-1c6c53000e5e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:50.507605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f957d-74ed-4ef0-a09c-1c6c53000e5e None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:50.511725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 702/1401] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:50.513957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e20e7-055e-4e72-b95d-77e1fa3bba05 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:50.514194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e20e7-055e-4e72-b95d-77e1fa3bba05 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:50.514496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e20e7-055e-4e72-b95d-77e1fa3bba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:50.593261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-4c3e20e7-055e-4e72-b95d-77e1fa3bba05 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:50.594415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4c3e20e7-055e-4e72-b95d-77e1fa3bba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:50.702068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 700/1402] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:50.991655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f957d-74ed-4ef0-a09c-1c6c53000e5e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:50.993198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7e0f957d-74ed-4ef0-a09c-1c6c53000e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:51.007929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:51.171644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f957d-74ed-4ef0-a09c-1c6c53000e5e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:51.278499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:51.312453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41dcd8-b77e-46d4-bb72-ae05a05eb43e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:51.313220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41dcd8-b77e-46d4-bb72-ae05a05eb43e service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:51.313497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41dcd8-b77e-46d4-bb72-ae05a05eb43e service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:51.313760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41dcd8-b77e-46d4-bb72-ae05a05eb43e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:51.315353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2c41dcd8-b77e-46d4-bb72-ae05a05eb43e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:51.424608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41dcd8-b77e-46d4-bb72-ae05a05eb43e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:51.426209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41dcd8-b77e-46d4-bb72-ae05a05eb43e service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:51.448327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 703/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:53.467230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57687a4-1ac0-4267-a096-73c591f83d5e None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:53.467839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57687a4-1ac0-4267-a096-73c591f83d5e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:53.467839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57687a4-1ac0-4267-a096-73c591f83d5e None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:53.469271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 701/1404] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:53.474243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f066f-d7be-416a-a286-82941162db8d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:53.474707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f066f-d7be-416a-a286-82941162db8d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:53.474864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f066f-d7be-416a-a286-82941162db8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:53.543844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-c36f066f-d7be-416a-a286-82941162db8d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:53.545421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c36f066f-d7be-416a-a286-82941162db8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:53.596565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:53.656328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 704/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:53.938973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57687a4-1ac0-4267-a096-73c591f83d5e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:53.940526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d57687a4-1ac0-4267-a096-73c591f83d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:54.003237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:54.108905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57687a4-1ac0-4267-a096-73c591f83d5e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:54.229581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1b42-3f04-44ce-94a0-864d0f132c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:54.230099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1b42-3f04-44ce-94a0-864d0f132c0b service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:54.230372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1b42-3f04-44ce-94a0-864d0f132c0b service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:54.230640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1b42-3f04-44ce-94a0-864d0f132c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:54.232235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-02ab1b42-3f04-44ce-94a0-864d0f132c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:54.246923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:54.331673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1b42-3f04-44ce-94a0-864d0f132c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:54.335990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1b42-3f04-44ce-94a0-864d0f132c0b service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:54.356738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 702/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:43:56.284792 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69b5cf-d6e2-4f72-8b9a-afb7fcae9504 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:56.285409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69b5cf-d6e2-4f72-8b9a-afb7fcae9504 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:56.285409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69b5cf-d6e2-4f72-8b9a-afb7fcae9504 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:56.286825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 705/1407] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:43:56.294195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10589-96b2-46a9-9398-b96146a2838b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:56.294707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10589-96b2-46a9-9398-b96146a2838b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:56.294707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10589-96b2-46a9-9398-b96146a2838b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:56.368853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-48c10589-96b2-46a9-9398-b96146a2838b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:43:56.370090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-48c10589-96b2-46a9-9398-b96146a2838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:56.466412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:56.481220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 703/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:43:56.767070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69b5cf-d6e2-4f72-8b9a-afb7fcae9504 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:43:56.768632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2e69b5cf-d6e2-4f72-8b9a-afb7fcae9504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:56.928653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69b5cf-d6e2-4f72-8b9a-afb7fcae9504 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:43:56.966558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:43:57.056852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b193dad-0ad0-4ee1-ab80-3f83a2e6d444 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:43:57.057549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b193dad-0ad0-4ee1-ab80-3f83a2e6d444 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:43:57.057888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b193dad-0ad0-4ee1-ab80-3f83a2e6d444 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:43:57.058222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b193dad-0ad0-4ee1-ab80-3f83a2e6d444 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:43:57.059914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3b193dad-0ad0-4ee1-ab80-3f83a2e6d444 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:43:57.175439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b193dad-0ad0-4ee1-ab80-3f83a2e6d444 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:43:57.176720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b193dad-0ad0-4ee1-ab80-3f83a2e6d444 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:43:57.199299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 706/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:44:06.476898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:07.196692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=83 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:11.982245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f567-8161-4d2d-aa92-18a0888b76ff None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:11.982856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f567-8161-4d2d-aa92-18a0888b76ff None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:11.982856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f567-8161-4d2d-aa92-18a0888b76ff None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:11.984231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 704/1410] 10.222.0.23 () {54 vars in 918 bytes} [Fri Aug 12 10:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:11.990435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086f7f6-cf58-409b-9974-4931b3b30a20 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:11.990888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086f7f6-cf58-409b-9974-4931b3b30a20 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:11.991029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086f7f6-cf58-409b-9974-4931b3b30a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:12.067029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1086f7f6-cf58-409b-9974-4931b3b30a20 None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:12.068299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1086f7f6-cf58-409b-9974-4931b3b30a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:12.187719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 707/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Fri Aug 12 10:44:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:12.203303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bbcf-60e8-4f5f-be16-df5d50695fe4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:12.203583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bbcf-60e8-4f5f-be16-df5d50695fe4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:12.203849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bbcf-60e8-4f5f-be16-df5d50695fe4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:12.205219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 705/1412] 10.222.0.23 () {54 vars in 918 bytes} [Fri Aug 12 10:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:12.213107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5962d5-bdd2-4f5e-b274-f3660b88d02b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:12.213471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5962d5-bdd2-4f5e-b274-f3660b88d02b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:12.213793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5962d5-bdd2-4f5e-b274-f3660b88d02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:12.293385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:12.296703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2b5962d5-bdd2-4f5e-b274-f3660b88d02b None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:12.298538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2b5962d5-bdd2-4f5e-b274-f3660b88d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:12.416221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 708/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Fri Aug 12 10:44:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:14.866677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb477323-2108-4f2c-b58e-16773fb2d721 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:14.867277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb477323-2108-4f2c-b58e-16773fb2d721 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:14.867277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb477323-2108-4f2c-b58e-16773fb2d721 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:14.868573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:14.875185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87502c-2ce8-4cda-b47b-7680ddb12527 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:14.875972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87502c-2ce8-4cda-b47b-7680ddb12527 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:14.876115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87502c-2ce8-4cda-b47b-7680ddb12527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:14.955292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1a87502c-2ce8-4cda-b47b-7680ddb12527 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:14.956553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1a87502c-2ce8-4cda-b47b-7680ddb12527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:15.025127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:15.069994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:15.357421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb477323-2108-4f2c-b58e-16773fb2d721 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:15.359174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-fb477323-2108-4f2c-b58e-16773fb2d721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:15.523305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb477323-2108-4f2c-b58e-16773fb2d721 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:15.605381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:15.656292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89514a1-d112-432c-bfba-2d0c1a68ff40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:15.657126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89514a1-d112-432c-bfba-2d0c1a68ff40 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:15.657488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89514a1-d112-432c-bfba-2d0c1a68ff40 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:15.657771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89514a1-d112-432c-bfba-2d0c1a68ff40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:15.660114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a89514a1-d112-432c-bfba-2d0c1a68ff40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:15.763713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89514a1-d112-432c-bfba-2d0c1a68ff40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:15.765361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89514a1-d112-432c-bfba-2d0c1a68ff40 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:15.784102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 707/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:16.671184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941b854-29b9-4c66-8685-27b94b3149c4 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:16.672075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941b854-29b9-4c66-8685-27b94b3149c4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:16.672075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941b854-29b9-4c66-8685-27b94b3149c4 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:16.674245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:16.680250 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ea460-3f09-4023-93ee-1183c8680a59 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:16.680943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ea460-3f09-4023-93ee-1183c8680a59 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:16.681160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ea460-3f09-4023-93ee-1183c8680a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:16.763698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-801ea460-3f09-4023-93ee-1183c8680a59 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:16.764974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-801ea460-3f09-4023-93ee-1183c8680a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:16.795623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:16.873781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:17.025277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941b854-29b9-4c66-8685-27b94b3149c4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:17.027420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6941b854-29b9-4c66-8685-27b94b3149c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:17.190866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941b854-29b9-4c66-8685-27b94b3149c4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:17.243694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:17.331069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943bb2a-3dcc-4ff4-ba0d-28ef219b2a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:17.331596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943bb2a-3dcc-4ff4-ba0d-28ef219b2a38 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:17.331829 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943bb2a-3dcc-4ff4-ba0d-28ef219b2a38 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:17.332135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943bb2a-3dcc-4ff4-ba0d-28ef219b2a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:17.333773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a943bb2a-3dcc-4ff4-ba0d-28ef219b2a38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:17.426460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943bb2a-3dcc-4ff4-ba0d-28ef219b2a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:17.428234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943bb2a-3dcc-4ff4-ba0d-28ef219b2a38 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:17.446932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 711/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:18.334294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35409b5f-e735-4c86-b75e-63babf2f1553 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:18.334928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35409b5f-e735-4c86-b75e-63babf2f1553 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:18.334928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35409b5f-e735-4c86-b75e-63babf2f1553 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:18.336578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:18.343179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f938e-f78c-4bff-bb92-e669e97163d0 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:18.343894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f938e-f78c-4bff-bb92-e669e97163d0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:18.344051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f938e-f78c-4bff-bb92-e669e97163d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:18.404411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:18.430458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-092f938e-f78c-4bff-bb92-e669e97163d0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:18.431671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-092f938e-f78c-4bff-bb92-e669e97163d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:18.532182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:18.689510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35409b5f-e735-4c86-b75e-63babf2f1553 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:18.692016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-35409b5f-e735-4c86-b75e-63babf2f1553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:18.879974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35409b5f-e735-4c86-b75e-63babf2f1553 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:18.899020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:19.032826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5018a73-1d19-4b87-8d41-50a9c730b1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:19.033329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5018a73-1d19-4b87-8d41-50a9c730b1f7 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:19.033602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5018a73-1d19-4b87-8d41-50a9c730b1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:19.033871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5018a73-1d19-4b87-8d41-50a9c730b1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:19.035491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a5018a73-1d19-4b87-8d41-50a9c730b1f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:19.129471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5018a73-1d19-4b87-8d41-50a9c730b1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:19.131222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5018a73-1d19-4b87-8d41-50a9c730b1f7 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:19.150927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 710/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:20.030162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a69c9a-0bf6-44e8-b196-f726996137c3 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:20.030768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a69c9a-0bf6-44e8-b196-f726996137c3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:20.030768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a69c9a-0bf6-44e8-b196-f726996137c3 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:20.032042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:20.038466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d87e7-23ea-4a26-93c5-dcd23ea2c87f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:20.039222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d87e7-23ea-4a26-93c5-dcd23ea2c87f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:20.039384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d87e7-23ea-4a26-93c5-dcd23ea2c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:20.064916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:20.121532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-439d87e7-23ea-4a26-93c5-dcd23ea2c87f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:20.122735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-439d87e7-23ea-4a26-93c5-dcd23ea2c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:20.223255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:20.373206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a69c9a-0bf6-44e8-b196-f726996137c3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:20.375591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d4a69c9a-0bf6-44e8-b196-f726996137c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:20.510903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:20.548348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a69c9a-0bf6-44e8-b196-f726996137c3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:20.680888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de1cae-cca5-4205-81c1-b7d62ec155bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:20.681495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de1cae-cca5-4205-81c1-b7d62ec155bb service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:20.681775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de1cae-cca5-4205-81c1-b7d62ec155bb service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:20.682055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de1cae-cca5-4205-81c1-b7d62ec155bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:20.683623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d6de1cae-cca5-4205-81c1-b7d62ec155bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:20.763603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:20.777484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de1cae-cca5-4205-81c1-b7d62ec155bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:20.779129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de1cae-cca5-4205-81c1-b7d62ec155bb service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:20.797606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 714/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:21.670070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa24256-ced1-4103-a09f-7fbeb8f2663f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:21.670930 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa24256-ced1-4103-a09f-7fbeb8f2663f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:21.670930 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa24256-ced1-4103-a09f-7fbeb8f2663f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:21.673056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:21.680434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9107fc-20a1-48a6-9af5-2a5930534a88 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:21.680880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9107fc-20a1-48a6-9af5-2a5930534a88 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:21.680972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9107fc-20a1-48a6-9af5-2a5930534a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:21.764428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-cc9107fc-20a1-48a6-9af5-2a5930534a88 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:21.765647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cc9107fc-20a1-48a6-9af5-2a5930534a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:21.872533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:44:22.030369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa24256-ced1-4103-a09f-7fbeb8f2663f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:22.032668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7fa24256-ced1-4103-a09f-7fbeb8f2663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:22.122885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:22.187597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa24256-ced1-4103-a09f-7fbeb8f2663f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:22.325146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2fc18-7a70-430f-8798-cfa5dceb647e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:22.325892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2fc18-7a70-430f-8798-cfa5dceb647e service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:22.326358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2fc18-7a70-430f-8798-cfa5dceb647e service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:22.326816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2fc18-7a70-430f-8798-cfa5dceb647e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:22.329855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d2e2fc18-7a70-430f-8798-cfa5dceb647e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:22.388590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:22.439315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2fc18-7a70-430f-8798-cfa5dceb647e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:22.440959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2fc18-7a70-430f-8798-cfa5dceb647e service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:22.462243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 713/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:23.282333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bc3eb-2c68-4d40-a94b-3eef59950d29 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:23.283046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bc3eb-2c68-4d40-a94b-3eef59950d29 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:23.283046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bc3eb-2c68-4d40-a94b-3eef59950d29 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:23.285001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:23.294435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d134-be6c-4545-b478-11048d7ce31f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:23.294875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d134-be6c-4545-b478-11048d7ce31f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:23.294997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d134-be6c-4545-b478-11048d7ce31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:23.378300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c193d134-be6c-4545-b478-11048d7ce31f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:23.379526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c193d134-be6c-4545-b478-11048d7ce31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:23.493409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:23.644838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bc3eb-2c68-4d40-a94b-3eef59950d29 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:23.646448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-014bc3eb-2c68-4d40-a94b-3eef59950d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:23.695637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:23.795302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bc3eb-2c68-4d40-a94b-3eef59950d29 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:23.910630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11d203-a4ab-44b6-92ab-9d9db3eb4f78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:23.911261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11d203-a4ab-44b6-92ab-9d9db3eb4f78 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:23.911534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11d203-a4ab-44b6-92ab-9d9db3eb4f78 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:23.911841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11d203-a4ab-44b6-92ab-9d9db3eb4f78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:23.913586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0f11d203-a4ab-44b6-92ab-9d9db3eb4f78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:23.920505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:24.024683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11d203-a4ab-44b6-92ab-9d9db3eb4f78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:24.026289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11d203-a4ab-44b6-92ab-9d9db3eb4f78 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:24.045329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 717/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:24.848301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddd74-bf9f-4e6c-8845-8fac00d4aa42 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:24.849153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddd74-bf9f-4e6c-8845-8fac00d4aa42 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:24.849153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eddd74-bf9f-4e6c-8845-8fac00d4aa42 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:24.850941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:24.856014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ac8ec-b67d-4e0c-ad67-644d27a1e8f4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:24.856501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ac8ec-b67d-4e0c-ad67-644d27a1e8f4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:24.856617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ac8ec-b67d-4e0c-ad67-644d27a1e8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:24.934886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-da8ac8ec-b67d-4e0c-ad67-644d27a1e8f4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:24.936064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-da8ac8ec-b67d-4e0c-ad67-644d27a1e8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:25.012070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:25.047585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:25.207565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eddd74-bf9f-4e6c-8845-8fac00d4aa42 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:25.210543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-94eddd74-bf9f-4e6c-8845-8fac00d4aa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:25.233966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:25.391430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eddd74-bf9f-4e6c-8845-8fac00d4aa42 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:25.498166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:25.532450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1edd24-949f-4dc0-a915-9f1371698252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:25.533058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1edd24-949f-4dc0-a915-9f1371698252 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:25.533324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1edd24-949f-4dc0-a915-9f1371698252 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:25.533584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1edd24-949f-4dc0-a915-9f1371698252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:25.535162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-bb1edd24-949f-4dc0-a915-9f1371698252 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:25.630824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1edd24-949f-4dc0-a915-9f1371698252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:25.632482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1edd24-949f-4dc0-a915-9f1371698252 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:25.652925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 716/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:26.453587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9001f-cc6f-4bbc-bb5a-46ebd02ef7a4 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:26.454481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9001f-cc6f-4bbc-bb5a-46ebd02ef7a4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:26.454481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9001f-cc6f-4bbc-bb5a-46ebd02ef7a4 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:26.456744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:26.462088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6050-6185-418b-847e-054e2590f995 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:26.462532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6050-6185-418b-847e-054e2590f995 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:26.462636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6050-6185-418b-847e-054e2590f995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:26.542555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5f6b6050-6185-418b-847e-054e2590f995 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:26.543753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5f6b6050-6185-418b-847e-054e2590f995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:26.587285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:26.645465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:44:26.794329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9001f-cc6f-4bbc-bb5a-46ebd02ef7a4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:26.796690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-dba9001f-cc6f-4bbc-bb5a-46ebd02ef7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:26.947787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9001f-cc6f-4bbc-bb5a-46ebd02ef7a4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:27.001600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:27.059982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb26b4-13e3-4924-8d47-22f0baf13325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:27.060643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb26b4-13e3-4924-8d47-22f0baf13325 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:27.060909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb26b4-13e3-4924-8d47-22f0baf13325 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:27.061209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb26b4-13e3-4924-8d47-22f0baf13325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:27.062904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-76eb26b4-13e3-4924-8d47-22f0baf13325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:27.156295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb26b4-13e3-4924-8d47-22f0baf13325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:27.157979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb26b4-13e3-4924-8d47-22f0baf13325 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:27.178512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 720/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:27.979011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68227c3b-7ba1-4753-bc52-7b588cab5176 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:27.979831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68227c3b-7ba1-4753-bc52-7b588cab5176 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:27.979831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68227c3b-7ba1-4753-bc52-7b588cab5176 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:27.982013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:44:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:27.989549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeafc07-8c78-44bf-be3e-13ee842ceeca None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:27.990042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeafc07-8c78-44bf-be3e-13ee842ceeca None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:27.990146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeafc07-8c78-44bf-be3e-13ee842ceeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:28.089306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7eeafc07-8c78-44bf-be3e-13ee842ceeca None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:28.090519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7eeafc07-8c78-44bf-be3e-13ee842ceeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:28.092570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:28.191049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:44:28.342435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68227c3b-7ba1-4753-bc52-7b588cab5176 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:28.344083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-68227c3b-7ba1-4753-bc52-7b588cab5176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:28.515184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68227c3b-7ba1-4753-bc52-7b588cab5176 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:28.544755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:28.638594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01ab0b-d522-4710-b56d-f770d424479a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:28.639118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01ab0b-d522-4710-b56d-f770d424479a service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:28.639382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01ab0b-d522-4710-b56d-f770d424479a service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:28.639642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01ab0b-d522-4710-b56d-f770d424479a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:28.641264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9d01ab0b-d522-4710-b56d-f770d424479a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:28.739311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01ab0b-d522-4710-b56d-f770d424479a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:28.740969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01ab0b-d522-4710-b56d-f770d424479a service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:28.759873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 719/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:30.579173 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d223d4-561f-41a9-a584-db88d532516a None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:30.580037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d223d4-561f-41a9-a584-db88d532516a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:30.580037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d223d4-561f-41a9-a584-db88d532516a None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:30.582244 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:30.587786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049915-e93b-4a56-87b8-668466c8e59d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:30.588222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049915-e93b-4a56-87b8-668466c8e59d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:30.588222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049915-e93b-4a56-87b8-668466c8e59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:30.633589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:30.662685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-43049915-e93b-4a56-87b8-668466c8e59d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:30.663901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-43049915-e93b-4a56-87b8-668466c8e59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:30.774358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:31.064695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d223d4-561f-41a9-a584-db88d532516a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:31.067123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-62d223d4-561f-41a9-a584-db88d532516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:31.199127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d223d4-561f-41a9-a584-db88d532516a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:31.233460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:31.279007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b6298-77ef-4385-a479-e2a0fadf61b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:31.279615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b6298-77ef-4385-a479-e2a0fadf61b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:31.279884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b6298-77ef-4385-a479-e2a0fadf61b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:31.280148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b6298-77ef-4385-a479-e2a0fadf61b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:31.281821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-266b6298-77ef-4385-a479-e2a0fadf61b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:31.398326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b6298-77ef-4385-a479-e2a0fadf61b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:31.399783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b6298-77ef-4385-a479-e2a0fadf61b5 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:31.418543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 723/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:33.452318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7094-aacd-4806-b916-db112757cd43 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:33.453270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7094-aacd-4806-b916-db112757cd43 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:33.453270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7094-aacd-4806-b916-db112757cd43 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:33.455241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:33.463851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57da80-8560-433e-81ea-e7db0f41f37c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:33.464361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57da80-8560-433e-81ea-e7db0f41f37c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:33.464463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57da80-8560-433e-81ea-e7db0f41f37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:33.535738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ee57da80-8560-433e-81ea-e7db0f41f37c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:33.537247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ee57da80-8560-433e-81ea-e7db0f41f37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:33.550782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:33.642544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:44:33.923106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b7094-aacd-4806-b916-db112757cd43 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:33.924663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-511b7094-aacd-4806-b916-db112757cd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:34.038608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b7094-aacd-4806-b916-db112757cd43 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:34.051759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:34.128247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68010d-27cb-437d-922e-9352d677ef12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:34.128890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68010d-27cb-437d-922e-9352d677ef12 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:34.129147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68010d-27cb-437d-922e-9352d677ef12 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:34.129434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68010d-27cb-437d-922e-9352d677ef12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:34.131187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4b68010d-27cb-437d-922e-9352d677ef12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:34.276380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68010d-27cb-437d-922e-9352d677ef12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:34.277797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68010d-27cb-437d-922e-9352d677ef12 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:34.299826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 722/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:44:36.654332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8620-e0e8-4937-b540-36aff5ded612 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:36.654917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8620-e0e8-4937-b540-36aff5ded612 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:36.654917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8620-e0e8-4937-b540-36aff5ded612 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:36.656232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:36.662765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8372719-0c3c-460b-9f24-28d5d17e82cb None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:36.663497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8372719-0c3c-460b-9f24-28d5d17e82cb None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:36.663657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8372719-0c3c-460b-9f24-28d5d17e82cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:36.721735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:36.741078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b8372719-0c3c-460b-9f24-28d5d17e82cb None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:36.742278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b8372719-0c3c-460b-9f24-28d5d17e82cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:36.840013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:37.129693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad8620-e0e8-4937-b540-36aff5ded612 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:37.131603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-dfad8620-e0e8-4937-b540-36aff5ded612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:37.258242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad8620-e0e8-4937-b540-36aff5ded612 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:37.309338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:37.348399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcc589-1cd2-4c97-b22b-b3a6762879fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:37.349043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcc589-1cd2-4c97-b22b-b3a6762879fb service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:37.349344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcc589-1cd2-4c97-b22b-b3a6762879fb service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:37.349610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcc589-1cd2-4c97-b22b-b3a6762879fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:37.351225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b8fcc589-1cd2-4c97-b22b-b3a6762879fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:37.476319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcc589-1cd2-4c97-b22b-b3a6762879fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:37.477805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcc589-1cd2-4c97-b22b-b3a6762879fb service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:37.500931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 726/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:39.712845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea35ae-d98f-43cd-997b-7c899d6f4ce2 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:39.713766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea35ae-d98f-43cd-997b-7c899d6f4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:39.713766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea35ae-d98f-43cd-997b-7c899d6f4ce2 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:39.715880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:39.721371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a316f-b433-4b1a-a666-f2cdc986ee5c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:39.721847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a316f-b433-4b1a-a666-f2cdc986ee5c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:39.721976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a316f-b433-4b1a-a666-f2cdc986ee5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:39.798382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7c6a316f-b433-4b1a-a666-f2cdc986ee5c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:39.799590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7c6a316f-b433-4b1a-a666-f2cdc986ee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:39.823080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:39.902927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:40.217306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea35ae-d98f-43cd-997b-7c899d6f4ce2 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:40.218866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-beea35ae-d98f-43cd-997b-7c899d6f4ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:40.347552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea35ae-d98f-43cd-997b-7c899d6f4ce2 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:40.372230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:40.436728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924de4d4-b9c9-48b1-8485-1ca485da270d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:40.437331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924de4d4-b9c9-48b1-8485-1ca485da270d service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:40.437605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924de4d4-b9c9-48b1-8485-1ca485da270d service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:40.437876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924de4d4-b9c9-48b1-8485-1ca485da270d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:40.439466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-924de4d4-b9c9-48b1-8485-1ca485da270d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:40.561878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924de4d4-b9c9-48b1-8485-1ca485da270d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:40.563257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924de4d4-b9c9-48b1-8485-1ca485da270d service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:40.582719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 725/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:42.538301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d6d9-7d55-4079-be88-359361ee7b67 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:42.538878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d6d9-7d55-4079-be88-359361ee7b67 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:42.538878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449d6d9-7d55-4079-be88-359361ee7b67 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:42.541094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:42.547937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a37b-0e2a-46ff-891e-b983554fcd86 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:42.548407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a37b-0e2a-46ff-891e-b983554fcd86 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:42.548536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a37b-0e2a-46ff-891e-b983554fcd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:42.614671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:42.618189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-75e4a37b-0e2a-46ff-891e-b983554fcd86 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:42.620143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-75e4a37b-0e2a-46ff-891e-b983554fcd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:42.717194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:43.001024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d6d9-7d55-4079-be88-359361ee7b67 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:43.003333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e449d6d9-7d55-4079-be88-359361ee7b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:43.112238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449d6d9-7d55-4079-be88-359361ee7b67 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:43.163876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:43.189565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab425e-9375-4852-9557-a7bfd13de502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:43.190171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab425e-9375-4852-9557-a7bfd13de502 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:43.190439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab425e-9375-4852-9557-a7bfd13de502 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:43.190702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab425e-9375-4852-9557-a7bfd13de502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:43.192300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-27ab425e-9375-4852-9557-a7bfd13de502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:43.307048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab425e-9375-4852-9557-a7bfd13de502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:43.308452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab425e-9375-4852-9557-a7bfd13de502 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:43.328057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 729/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:45.275074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ffb03-6e2b-49c3-80bc-aa56525e7a41 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:45.275579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ffb03-6e2b-49c3-80bc-aa56525e7a41 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:45.275693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ffb03-6e2b-49c3-80bc-aa56525e7a41 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:45.277181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:45.283363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb079-ce70-401a-8474-1e0dfed22e97 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:45.283795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb079-ce70-401a-8474-1e0dfed22e97 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:45.283906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb079-ce70-401a-8474-1e0dfed22e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:45.354727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3cdbb079-ce70-401a-8474-1e0dfed22e97 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:45.355940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3cdbb079-ce70-401a-8474-1e0dfed22e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:45.386051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:45.456288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:45.736708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ffb03-6e2b-49c3-80bc-aa56525e7a41 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:45.738253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f85ffb03-6e2b-49c3-80bc-aa56525e7a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:45.844819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ffb03-6e2b-49c3-80bc-aa56525e7a41 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:45.876334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:45.922872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b80142-68d4-4d7e-a840-d7efc1b4a705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:45.923470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b80142-68d4-4d7e-a840-d7efc1b4a705 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:45.923736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b80142-68d4-4d7e-a840-d7efc1b4a705 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:45.923972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b80142-68d4-4d7e-a840-d7efc1b4a705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:45.925603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-19b80142-68d4-4d7e-a840-d7efc1b4a705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:46.054736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b80142-68d4-4d7e-a840-d7efc1b4a705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:46.056138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b80142-68d4-4d7e-a840-d7efc1b4a705 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:46.076309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 728/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:48.002542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893132b7-35aa-482f-afe0-84e12e6f6f18 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:48.003451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893132b7-35aa-482f-afe0-84e12e6f6f18 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:48.003451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893132b7-35aa-482f-afe0-84e12e6f6f18 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:48.006786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:48.016629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f64ec0-83bc-4d48-a611-ba5e285715ad None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:48.017100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f64ec0-83bc-4d48-a611-ba5e285715ad None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:48.017204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f64ec0-83bc-4d48-a611-ba5e285715ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:48.100441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-99f64ec0-83bc-4d48-a611-ba5e285715ad None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:48.101651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-99f64ec0-83bc-4d48-a611-ba5e285715ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:48.110462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:48.203723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:48.485308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893132b7-35aa-482f-afe0-84e12e6f6f18 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:48.486879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-893132b7-35aa-482f-afe0-84e12e6f6f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:48.601280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893132b7-35aa-482f-afe0-84e12e6f6f18 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:48.667135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:48.687344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d3b7-9aeb-46bc-b980-de18c42b659a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:48.687946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d3b7-9aeb-46bc-b980-de18c42b659a service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:48.688215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d3b7-9aeb-46bc-b980-de18c42b659a service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:48.688509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d3b7-9aeb-46bc-b980-de18c42b659a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:48.690092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5331d3b7-9aeb-46bc-b980-de18c42b659a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:48.802071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d3b7-9aeb-46bc-b980-de18c42b659a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:48.803700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d3b7-9aeb-46bc-b980-de18c42b659a service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:48.821752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 732/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:44:50.756836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba407a-86c3-4525-acb6-5c6636fc4bd4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:50.757475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba407a-86c3-4525-acb6-5c6636fc4bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:50.757475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba407a-86c3-4525-acb6-5c6636fc4bd4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:50.758756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:50.767525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6deb9-bc41-4039-aec4-30184ee713c3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:50.768241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6deb9-bc41-4039-aec4-30184ee713c3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:50.768241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6deb9-bc41-4039-aec4-30184ee713c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:50.843759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d0d6deb9-bc41-4039-aec4-30184ee713c3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:50.845284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d0d6deb9-bc41-4039-aec4-30184ee713c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:50.885020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:50.951939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:51.234390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba407a-86c3-4525-acb6-5c6636fc4bd4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:51.236839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f3ba407a-86c3-4525-acb6-5c6636fc4bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:51.360425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba407a-86c3-4525-acb6-5c6636fc4bd4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:51.402579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:51.449887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b2cfe-3068-463b-bfc8-b262a0e493b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:51.450500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b2cfe-3068-463b-bfc8-b262a0e493b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:51.450778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b2cfe-3068-463b-bfc8-b262a0e493b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:51.451039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b2cfe-3068-463b-bfc8-b262a0e493b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:51.452670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ac0b2cfe-3068-463b-bfc8-b262a0e493b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:51.566113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b2cfe-3068-463b-bfc8-b262a0e493b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:51.567476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b2cfe-3068-463b-bfc8-b262a0e493b3 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:51.587145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 731/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:53.544770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1a189-c9fb-454a-9492-4f53ae93c115 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:53.545360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1a189-c9fb-454a-9492-4f53ae93c115 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:53.545360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1a189-c9fb-454a-9492-4f53ae93c115 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:53.547390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:53.553433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01bf6c-2e0c-466c-a74f-9fbd6365c9f6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:53.553890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01bf6c-2e0c-466c-a74f-9fbd6365c9f6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:53.554009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01bf6c-2e0c-466c-a74f-9fbd6365c9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:53.623951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ba01bf6c-2e0c-466c-a74f-9fbd6365c9f6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:53.625090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ba01bf6c-2e0c-466c-a74f-9fbd6365c9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:53.645095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:53.726949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:54.012787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1a189-c9fb-454a-9492-4f53ae93c115 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:54.015487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c4c1a189-c9fb-454a-9492-4f53ae93c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:54.132955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1a189-c9fb-454a-9492-4f53ae93c115 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:54.210047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:54.218425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1bca0-149e-41b9-9b38-0cfab63b0a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:54.219029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1bca0-149e-41b9-9b38-0cfab63b0a66 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:54.219295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1bca0-149e-41b9-9b38-0cfab63b0a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:54.219561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1bca0-149e-41b9-9b38-0cfab63b0a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:54.221198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f2a1bca0-149e-41b9-9b38-0cfab63b0a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:54.363531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1bca0-149e-41b9-9b38-0cfab63b0a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:54.365046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1bca0-149e-41b9-9b38-0cfab63b0a66 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:54.387905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 735/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:44:56.313973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501e181-780c-4d6c-9101-d0cad0e6cdc1 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:56.314599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501e181-780c-4d6c-9101-d0cad0e6cdc1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:56.314599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501e181-780c-4d6c-9101-d0cad0e6cdc1 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:56.316699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:44:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:44:56.325478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e7bdd-5703-40ec-9952-7203fe3d514d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:56.326234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e7bdd-5703-40ec-9952-7203fe3d514d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:56.326339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e7bdd-5703-40ec-9952-7203fe3d514d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:56.399504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-e71e7bdd-5703-40ec-9952-7203fe3d514d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:44:56.400742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e71e7bdd-5703-40ec-9952-7203fe3d514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:56.452768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:56.508446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:44:56.788941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501e181-780c-4d6c-9101-d0cad0e6cdc1 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:44:56.790529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a501e181-780c-4d6c-9101-d0cad0e6cdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:56.900537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501e181-780c-4d6c-9101-d0cad0e6cdc1 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:44:56.955240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:44:56.983559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916eed9a-e8c6-41a7-8422-1c5a8ab63422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:44:56.984166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eed9a-e8c6-41a7-8422-1c5a8ab63422 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:44:56.984473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eed9a-e8c6-41a7-8422-1c5a8ab63422 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:44:56.984740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eed9a-e8c6-41a7-8422-1c5a8ab63422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:44:56.986346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-916eed9a-e8c6-41a7-8422-1c5a8ab63422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:44:57.120796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916eed9a-e8c6-41a7-8422-1c5a8ab63422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:44:57.122192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916eed9a-e8c6-41a7-8422-1c5a8ab63422 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:44:57.143689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 734/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:45:06.506516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=22 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:07.140754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=66 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:16.716972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283829-959f-429c-8905-ba8f8aaa305d None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:16.717533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283829-959f-429c-8905-ba8f8aaa305d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:16.717533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43283829-959f-429c-8905-ba8f8aaa305d None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:16.719678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 737/1471] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:45:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:45:16.728582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c9eee-baac-4a74-8076-a284c5232931 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:16.729038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c9eee-baac-4a74-8076-a284c5232931 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:16.729137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c9eee-baac-4a74-8076-a284c5232931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:16.811980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9d5c9eee-baac-4a74-8076-a284c5232931 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:16.813916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9d5c9eee-baac-4a74-8076-a284c5232931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:16.923713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 735/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:45:17.239385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886341d9-ab39-49ea-bacd-fbed1af05720 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:17.239774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886341d9-ab39-49ea-bacd-fbed1af05720 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:17.240151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886341d9-ab39-49ea-bacd-fbed1af05720 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:17.242589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 738/1473] 10.222.0.23 () {54 vars in 967 bytes} [Fri Aug 12 10:45:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:45:17.247743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cec13-64a7-40f2-b479-f9f76012644a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:17.248026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cec13-64a7-40f2-b479-f9f76012644a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:17.248291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cec13-64a7-40f2-b479-f9f76012644a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:17.326993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-683cec13-64a7-40f2-b479-f9f76012644a None None] MFA Rules not processed for user `8571f3d80b0749729012353fcbbd9e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:17.328197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-683cec13-64a7-40f2-b479-f9f76012644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:17.350188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:17.417889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 736/1474] 10.222.0.23 () {58 vars in 1064 bytes} [Fri Aug 12 10:45:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:45:17.425643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886341d9-ab39-49ea-bacd-fbed1af05720 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:45:17.427161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-886341d9-ab39-49ea-bacd-fbed1af05720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:17.543577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886341d9-ab39-49ea-bacd-fbed1af05720 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:45:17.621561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee309363-ab0b-4901-851b-0f1daf1e714a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8571f3d80b0749729012353fcbbd9e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:45:17.622162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309363-ab0b-4901-851b-0f1daf1e714a service glance] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:17.622435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309363-ab0b-4901-851b-0f1daf1e714a service glance] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:17.622702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309363-ab0b-4901-851b-0f1daf1e714a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:17.624301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ee309363-ab0b-4901-851b-0f1daf1e714a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:17.679804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:17.739076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee309363-ab0b-4901-851b-0f1daf1e714a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:45:17.740544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee309363-ab0b-4901-851b-0f1daf1e714a service glance] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:45:17.761004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 739/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:45:19.751735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e745f9-a241-47f4-be7f-0d62c4ba8be6 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:19.752683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e745f9-a241-47f4-be7f-0d62c4ba8be6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:19.752683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e745f9-a241-47f4-be7f-0d62c4ba8be6 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:19.754855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 737/1476] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:45:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:45:19.760518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4327a-0c53-4f8e-8bbb-87edad37ca83 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:19.760958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4327a-0c53-4f8e-8bbb-87edad37ca83 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:19.761064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4327a-0c53-4f8e-8bbb-87edad37ca83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:19.962225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7de4327a-0c53-4f8e-8bbb-87edad37ca83 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:19.963423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7de4327a-0c53-4f8e-8bbb-87edad37ca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:20.066765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 740/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:45:20.334394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e745f9-a241-47f4-be7f-0d62c4ba8be6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:45:20.336224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-35e745f9-a241-47f4-be7f-0d62c4ba8be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:20.473840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e745f9-a241-47f4-be7f-0d62c4ba8be6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:45:20.533578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:20.580839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66d87b-27f6-45a6-8116-0055ea6f7da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:45:20.581442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66d87b-27f6-45a6-8116-0055ea6f7da8 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:20.581712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66d87b-27f6-45a6-8116-0055ea6f7da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:20.581976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66d87b-27f6-45a6-8116-0055ea6f7da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:20.582639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66d87b-27f6-45a6-8116-0055ea6f7da8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:45:20.584511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-6f66d87b-27f6-45a6-8116-0055ea6f7da8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:45:20.584511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:45:20.584511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:45:20.585657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66d87b-27f6-45a6-8116-0055ea6f7da8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:45:20.607005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 738/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Aug 12 10:45:20 2022] GET /identity/v3/projects => generated 2553 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:45:22.607368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c87a-f237-404e-8eee-6a2aaa1c10fb None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:22.608106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c87a-f237-404e-8eee-6a2aaa1c10fb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:22.608106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c87a-f237-404e-8eee-6a2aaa1c10fb None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:22.609884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 741/1479] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:45:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:45:22.615501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e7163-c19e-46bc-b64c-2b6d21375e07 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:22.615908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e7163-c19e-46bc-b64c-2b6d21375e07 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:22.616030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e7163-c19e-46bc-b64c-2b6d21375e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:22.688285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f19e7163-c19e-46bc-b64c-2b6d21375e07 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:22.689541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f19e7163-c19e-46bc-b64c-2b6d21375e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:22.803155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 739/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:45:23.107944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475c87a-f237-404e-8eee-6a2aaa1c10fb None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:45:23.109908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5475c87a-f237-404e-8eee-6a2aaa1c10fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:23.120883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:23.266032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475c87a-f237-404e-8eee-6a2aaa1c10fb None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:45:23.360393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:23.395149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad122ab-3602-4c4f-b3bf-83f87d51c0d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:45:23.395737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad122ab-3602-4c4f-b3bf-83f87d51c0d2 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:23.396017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad122ab-3602-4c4f-b3bf-83f87d51c0d2 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:23.396289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad122ab-3602-4c4f-b3bf-83f87d51c0d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:23.397916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bad122ab-3602-4c4f-b3bf-83f87d51c0d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:23.502926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad122ab-3602-4c4f-b3bf-83f87d51c0d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:45:23.504557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad122ab-3602-4c4f-b3bf-83f87d51c0d2 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:45:23.525127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 742/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:45:25.619309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11449370-1738-4066-85ba-d5c9632ba5c3 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:25.619938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11449370-1738-4066-85ba-d5c9632ba5c3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:25.619938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11449370-1738-4066-85ba-d5c9632ba5c3 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:25.621158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 740/1482] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:45:25.627550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697a0de-da56-4d5c-9a8f-1b1ee394593b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:25.628299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697a0de-da56-4d5c-9a8f-1b1ee394593b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:25.628498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697a0de-da56-4d5c-9a8f-1b1ee394593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:25.705651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0697a0de-da56-4d5c-9a8f-1b1ee394593b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:25.706901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0697a0de-da56-4d5c-9a8f-1b1ee394593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:25.754150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:25.808676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 743/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:45:26.107711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11449370-1738-4066-85ba-d5c9632ba5c3 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:45:26.110269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-11449370-1738-4066-85ba-d5c9632ba5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:26.138084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:26.224637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11449370-1738-4066-85ba-d5c9632ba5c3 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:45:26.301995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d09d3-48a0-48f3-84fc-5ebf4b2f9c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:45:26.302573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d09d3-48a0-48f3-84fc-5ebf4b2f9c71 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:26.302884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d09d3-48a0-48f3-84fc-5ebf4b2f9c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:26.303149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d09d3-48a0-48f3-84fc-5ebf4b2f9c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:26.304802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-014d09d3-48a0-48f3-84fc-5ebf4b2f9c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:26.389554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:26.426229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d09d3-48a0-48f3-84fc-5ebf4b2f9c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:45:26.427680 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d09d3-48a0-48f3-84fc-5ebf4b2f9c71 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:45:26.447548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 741/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:45:28.422226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d254-bf01-4429-88eb-bcef34e0f977 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:28.422782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d254-bf01-4429-88eb-bcef34e0f977 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:28.422782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0d254-bf01-4429-88eb-bcef34e0f977 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:28.424097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 744/1485] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:45:28.430493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117cbdd-2edc-4f5d-9203-399c44976937 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:28.430901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117cbdd-2edc-4f5d-9203-399c44976937 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:28.431017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117cbdd-2edc-4f5d-9203-399c44976937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:28.504917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e117cbdd-2edc-4f5d-9203-399c44976937 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:28.506130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e117cbdd-2edc-4f5d-9203-399c44976937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:28.610077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 742/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:45:28.901260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d254-bf01-4429-88eb-bcef34e0f977 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:45:28.903682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-46f0d254-bf01-4429-88eb-bcef34e0f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:29.028052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0d254-bf01-4429-88eb-bcef34e0f977 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:45:29.109928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:29.113325 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f54ad2-148a-4c42-90a1-66f28bdb9a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:45:29.114283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f54ad2-148a-4c42-90a1-66f28bdb9a5f service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:29.114912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f54ad2-148a-4c42-90a1-66f28bdb9a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:29.115201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f54ad2-148a-4c42-90a1-66f28bdb9a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:29.117431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-03f54ad2-148a-4c42-90a1-66f28bdb9a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:29.239484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f54ad2-148a-4c42-90a1-66f28bdb9a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:45:29.240986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f54ad2-148a-4c42-90a1-66f28bdb9a5f service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:45:29.265703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 745/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:45:31.268750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fdabe-33e4-4ad6-8e73-b28c90fe6ed4 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:31.269368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fdabe-33e4-4ad6-8e73-b28c90fe6ed4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:31.269368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fdabe-33e4-4ad6-8e73-b28c90fe6ed4 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:31.270647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 743/1488] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:45:31.277513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf37334-114d-4176-a68f-05fc746b3116 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:31.278299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf37334-114d-4176-a68f-05fc746b3116 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:31.278299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf37334-114d-4176-a68f-05fc746b3116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:31.360157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-abf37334-114d-4176-a68f-05fc746b3116 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:31.361390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-abf37334-114d-4176-a68f-05fc746b3116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:31.415611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:31.461690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 746/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:45:31.756535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fdabe-33e4-4ad6-8e73-b28c90fe6ed4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:45:31.758166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e21fdabe-33e4-4ad6-8e73-b28c90fe6ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:31.801115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:31.872154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fdabe-33e4-4ad6-8e73-b28c90fe6ed4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:45:31.956664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a0a9e-bfa0-4fb1-9482-9c19494afcec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:45:31.957264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a0a9e-bfa0-4fb1-9482-9c19494afcec service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:31.957535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a0a9e-bfa0-4fb1-9482-9c19494afcec service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:31.957799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a0a9e-bfa0-4fb1-9482-9c19494afcec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:31.959366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-af3a0a9e-bfa0-4fb1-9482-9c19494afcec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:32.054499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:32.085554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a0a9e-bfa0-4fb1-9482-9c19494afcec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:45:32.087261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a0a9e-bfa0-4fb1-9482-9c19494afcec service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:45:32.105790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 744/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:45:34.230138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389ce0-4e40-4ef1-957d-8caa5dfa6514 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:34.231044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389ce0-4e40-4ef1-957d-8caa5dfa6514 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:34.231044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389ce0-4e40-4ef1-957d-8caa5dfa6514 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:34.233267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 747/1491] 10.222.0.23 () {54 vars in 927 bytes} [Fri Aug 12 10:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:45:34.238722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef250d3-3516-40d0-a90b-695f9933d571 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:34.239175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef250d3-3516-40d0-a90b-695f9933d571 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:34.239281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef250d3-3516-40d0-a90b-695f9933d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:34.312584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5ef250d3-3516-40d0-a90b-695f9933d571 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:45:34.314417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5ef250d3-3516-40d0-a90b-695f9933d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:34.422359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 745/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Fri Aug 12 10:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:45:34.722446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389ce0-4e40-4ef1-957d-8caa5dfa6514 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:45:34.724971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ab389ce0-4e40-4ef1-957d-8caa5dfa6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:34.834742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389ce0-4e40-4ef1-957d-8caa5dfa6514 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:45:34.912839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5046e72-0206-4cf4-84db-a2e1d9fc5156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:45:34.913456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5046e72-0206-4cf4-84db-a2e1d9fc5156 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:45:34.913721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5046e72-0206-4cf4-84db-a2e1d9fc5156 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:45:34.913988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5046e72-0206-4cf4-84db-a2e1d9fc5156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:45:34.915724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c5046e72-0206-4cf4-84db-a2e1d9fc5156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:45:34.925458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:35.045196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5046e72-0206-4cf4-84db-a2e1d9fc5156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:45:35.046668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5046e72-0206-4cf4-84db-a2e1d9fc5156 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:45:35.067739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 748/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:45:44.420615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=82 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:45:45.065440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=50 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:46.643284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:46.643875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:46.643875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:46.720133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:46:46.721354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:46.820186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 746/1494] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:46:46.828875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a8c7-2dde-4948-847b-55f0631fef28 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:46.829366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a8c7-2dde-4948-847b-55f0631fef28 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:46.829477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a8c7-2dde-4948-847b-55f0631fef28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:46.920801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1bd1a8c7-2dde-4948-847b-55f0631fef28 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:46:46.921991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1bd1a8c7-2dde-4948-847b-55f0631fef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:47.023095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 749/1495] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:46:47.029082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:47.030872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:47.050215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a35d74-080e-436a-b512-6ee952a9b739 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:47.126160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:47.162605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:47.163202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:47.163470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:47.163732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:47.164524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:46:47.164823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:47.165840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:47.165840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:46:47.165840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:46:47.166735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:47.178198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 747/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:46:47.186910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a8c7-2dde-4948-847b-55f0631fef28 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:47.196535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a8c7-2dde-4948-847b-55f0631fef28 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:47.262270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:47.297611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:47.298209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:47.298481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:47.298745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:47.299358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:47.301142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING py.warnings [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:47.301142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: warnings.warn(msg) Aug 12 10:46:47.301142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: Aug 12 10:46:47.302327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:47.362264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 750/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:46:47 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:46:47.366590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:47.368156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:47.403319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990650-1aca-4863-805b-f12230a2ed21 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:47.504869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:47.505396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:47.505665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:47.505947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:47.506555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:47.507575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:47.507575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:46:47.507575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:46:47.508964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:47.557248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 748/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:46:47 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:46:47.561566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:47.563066 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:47.627791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d466384-5a4f-4612-a6ae-811c8e59fc36 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:47.719343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:47.730926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:47.731535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:47.731806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:47.732084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:47.732743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:47.733769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING py.warnings [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:47.733769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: warnings.warn(msg) Aug 12 10:46:47.733769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: Aug 12 10:46:47.734645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:47.747706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 751/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:46:47.751997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:47.753717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:47.765756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe21026-953d-482d-b72e-afc094177d03 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:47.791230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:47.875498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58724b-53bc-466f-8069-bc423a1f0464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:47.876106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58724b-53bc-466f-8069-bc423a1f0464 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:47.876406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58724b-53bc-466f-8069-bc423a1f0464 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:47.876667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58724b-53bc-466f-8069-bc423a1f0464 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:47.877302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58724b-53bc-466f-8069-bc423a1f0464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:47.878320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-5b58724b-53bc-466f-8069-bc423a1f0464 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:47.878320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:46:47.878320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:46:47.879198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58724b-53bc-466f-8069-bc423a1f0464 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:47.893113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 749/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:46:47.898760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:47.900842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:47.934937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b082471-4e19-4644-b91f-4f26181d096e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:48.035744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6fadf-71de-4daa-b821-ea61233b3897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:48.036316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fadf-71de-4daa-b821-ea61233b3897 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:48.036606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fadf-71de-4daa-b821-ea61233b3897 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:48.036869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fadf-71de-4daa-b821-ea61233b3897 admin admin] PATH_INFO: `/v3/projects/5c35168a5f734228b3b955a5f56f6eda/users/5b330bca0cf44e9b98f80326f1bcbeae/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:48.060858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6fadf-71de-4daa-b821-ea61233b3897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c35168a5f734228b3b955a5f56f6eda, user_id=5b330bca0cf44e9b98f80326f1bcbeae, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:48.061884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING py.warnings [None req-3ab6fadf-71de-4daa-b821-ea61233b3897 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:48.061884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: warnings.warn(msg) Aug 12 10:46:48.061884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: Aug 12 10:46:48.063838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6fadf-71de-4daa-b821-ea61233b3897 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:48.091310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 752/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:46:47 2022] PUT /identity/v3/projects/5c35168a5f734228b3b955a5f56f6eda/users/5b330bca0cf44e9b98f80326f1bcbeae/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:46:48.103915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300825-6558-4596-98e6-a773c14c458c None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:48.104218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300825-6558-4596-98e6-a773c14c458c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:48.104515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300825-6558-4596-98e6-a773c14c458c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:48.165174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a3300825-6558-4596-98e6-a773c14c458c None None] MFA Rules not processed for user `5b330bca0cf44e9b98f80326f1bcbeae`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:46:48.166351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a3300825-6558-4596-98e6-a773c14c458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:48.252197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 750/1502] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:46:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:46:48.271885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae230-b544-4ade-929f-c4379bdcd4cb None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:48.272238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae230-b544-4ade-929f-c4379bdcd4cb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:48.272540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae230-b544-4ade-929f-c4379bdcd4cb None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:48.273973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 753/1503] 10.222.0.23 () {54 vars in 964 bytes} [Fri Aug 12 10:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:46:48.278972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8fae-f67b-452d-b7f1-d96bfa545410 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:48.279254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8fae-f67b-452d-b7f1-d96bfa545410 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:48.279522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8fae-f67b-452d-b7f1-d96bfa545410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:48.315117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:48.364649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f39b8fae-f67b-452d-b7f1-d96bfa545410 None None] MFA Rules not processed for user `76f4fb448f1443b7a77fd8e1954defe5`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:46:48.365841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f39b8fae-f67b-452d-b7f1-d96bfa545410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:48.447393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 751/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Aug 12 10:46:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:46:48.456095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ae230-b544-4ade-929f-c4379bdcd4cb None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:48.458514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d97ae230-b544-4ade-929f-c4379bdcd4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:48.510733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:48.568908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ae230-b544-4ade-929f-c4379bdcd4cb None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:48.651397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:48.652019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:48.652300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:48.652575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:48.654155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:48.776596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:48.778082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:48.781538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:48.797094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 754/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:46:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:46:48.847106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ba4c5-61ca-45bd-a872-9fc1023f4d79 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:48.847520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ba4c5-61ca-45bd-a872-9fc1023f4d79 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:48.847658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ba4c5-61ca-45bd-a872-9fc1023f4d79 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:48.849514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 752/1506] 10.222.0.23 () {52 vars in 830 bytes} [Fri Aug 12 10:46:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:46:48.854908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:48.856887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:48.968435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f5d9-dbe4-4d73-931f-89acca53a15e service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:49.092669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:49.093208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:49.093473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:49.093733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] PATH_INFO: `/v3/users/5b330bca0cf44e9b98f80326f1bcbeae` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:49.107995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b330bca0cf44e9b98f80326f1bcbeae)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:49.109071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING py.warnings [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:49.109071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: warnings.warn(msg) Aug 12 10:46:49.109071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: Aug 12 10:46:49.110273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:49.120634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:49.184968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b330bca0cf44e9b98f80326f1bcbeae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:49.199468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b330bca0cf44e9b98f80326f1bcbeae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:49.200045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b330bca0cf44e9b98f80326f1bcbeae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:49.211158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b330bca0cf44e9b98f80326f1bcbeae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:49.235749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b330bca0cf44e9b98f80326f1bcbeae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:49.242010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 755/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:46:48 2022] DELETE /identity/v3/users/5b330bca0cf44e9b98f80326f1bcbeae => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:46:49.247301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:49.247551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:49.247821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:49.305825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:46:49.307025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:49.386080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:49.399863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 753/1508] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:46:49.409749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:49.410967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:49.519949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03213b-b608-4a27-b0e5-b8b3b7b12f3c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:49.577280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:46:49.599715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf400f46-56ca-4e99-b5d0-7b53dd7b986b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:49.600352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf400f46-56ca-4e99-b5d0-7b53dd7b986b service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:49.600631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf400f46-56ca-4e99-b5d0-7b53dd7b986b service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:49.600885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf400f46-56ca-4e99-b5d0-7b53dd7b986b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:49.603128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bf400f46-56ca-4e99-b5d0-7b53dd7b986b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:49.722758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf400f46-56ca-4e99-b5d0-7b53dd7b986b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:49.724203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf400f46-56ca-4e99-b5d0-7b53dd7b986b service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:49.743456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 756/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:46:50.091319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:46:50.092797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:46:50.106618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cbd46-d9cd-407b-bcc3-366fcd25ce0b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:46:50.203792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:46:50.204466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:46:50.204740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:46:50.205002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] PATH_INFO: `/v3/projects/5c35168a5f734228b3b955a5f56f6eda` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:46:50.215325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c35168a5f734228b3b955a5f56f6eda)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:46:50.216400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:46:50.216400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:46:50.216400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:46:50.217602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:46:50.290457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c35168a5f734228b3b955a5f56f6eda', 'request_id': 'req-c185b78b-e378-4087-87ea-5331a6afcc3c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:50.303504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c35168a5f734228b3b955a5f56f6eda', 'request_id': 'req-c185b78b-e378-4087-87ea-5331a6afcc3c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:50.310526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c35168a5f734228b3b955a5f56f6eda', 'request_id': 'req-c185b78b-e378-4087-87ea-5331a6afcc3c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:50.340233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] The token cache is being invalidate because project 5c35168a5f734228b3b955a5f56f6eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:46:50.340544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:50.340974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c185b78b-e378-4087-87ea-5331a6afcc3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:46:50.342440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 754/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:46:50 2022] DELETE /identity/v3/projects/5c35168a5f734228b3b955a5f56f6eda => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:46:59.740641 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=62 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:47:00.338037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:48:18.962076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd6113-0853-4708-b623-e11262edcdea None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:18.962686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd6113-0853-4708-b623-e11262edcdea None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:18.962686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd6113-0853-4708-b623-e11262edcdea None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:18.964613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:18.971550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5835-9ddb-44cb-9e4d-1a1a94b26ce1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:18.972016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5835-9ddb-44cb-9e4d-1a1a94b26ce1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:18.972119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5835-9ddb-44cb-9e4d-1a1a94b26ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:19.055589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-612e5835-9ddb-44cb-9e4d-1a1a94b26ce1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:19.056820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-612e5835-9ddb-44cb-9e4d-1a1a94b26ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:19.179099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:19.345578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd6113-0853-4708-b623-e11262edcdea None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:19.349026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d9cd6113-0853-4708-b623-e11262edcdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:19.486966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd6113-0853-4708-b623-e11262edcdea None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:19.493245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a489bb-71fd-4562-a874-541d81f3a677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:19.493846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a489bb-71fd-4562-a874-541d81f3a677 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:19.494120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a489bb-71fd-4562-a874-541d81f3a677 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:19.494371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a489bb-71fd-4562-a874-541d81f3a677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:19.502526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a489bb-71fd-4562-a874-541d81f3a677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:19.504158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a489bb-71fd-4562-a874-541d81f3a677 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:19.513628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 758/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:48:20.412421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323acc1-9b4b-4f99-819b-3a17480f4b37 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:20.413003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323acc1-9b4b-4f99-819b-3a17480f4b37 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:20.413003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323acc1-9b4b-4f99-819b-3a17480f4b37 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:20.414478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:20.420038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716192d-16dc-41b1-a7ee-09af05f0c857 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:20.420550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716192d-16dc-41b1-a7ee-09af05f0c857 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:20.420651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716192d-16dc-41b1-a7ee-09af05f0c857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:20.446447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b716192d-16dc-41b1-a7ee-09af05f0c857 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:20.447700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b716192d-16dc-41b1-a7ee-09af05f0c857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:20.461654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:20.492947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c300-4b83-4787-bc6c-e1cb813309ad None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:20.493302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c300-4b83-4787-bc6c-e1cb813309ad None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:20.493815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70c300-4b83-4787-bc6c-e1cb813309ad None None] PATH_INFO: `/v3` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:20.494988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 757/1516] 10.222.0.39 () {54 vars in 925 bytes} [Fri Aug 12 10:48:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:48:20.502335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8e9cd-6468-4623-94ad-26098414a375 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:20.502682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8e9cd-6468-4623-94ad-26098414a375 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:20.502952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8e9cd-6468-4623-94ad-26098414a375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:20.568221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-eae8e9cd-6468-4623-94ad-26098414a375 None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:20.569546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-eae8e9cd-6468-4623-94ad-26098414a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:20.628896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:48:20.640419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c300-4b83-4787-bc6c-e1cb813309ad None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:20.651415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 760/1517] 10.222.0.39 () {58 vars in 1015 bytes} [Fri Aug 12 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:20.656122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70c300-4b83-4787-bc6c-e1cb813309ad None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:20.667445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4be147-d9c5-431c-a67b-133b6ac64328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:20.668050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4be147-d9c5-431c-a67b-133b6ac64328 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:20.668359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4be147-d9c5-431c-a67b-133b6ac64328 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:20.668594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4be147-d9c5-431c-a67b-133b6ac64328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:20.679128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4be147-d9c5-431c-a67b-133b6ac64328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:20.681238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4be147-d9c5-431c-a67b-133b6ac64328 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:20.691237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 758/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:48:29.485552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c547a7-e98a-4adc-b751-c43fd23e3aff None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:29.486097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c547a7-e98a-4adc-b751-c43fd23e3aff None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:29.486097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c547a7-e98a-4adc-b751-c43fd23e3aff None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:29.487287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 761/1519] 10.222.0.23 () {54 vars in 961 bytes} [Fri Aug 12 10:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:29.493235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a4a51-1b0c-4867-bf22-4040cc08a0f5 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:29.493659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a4a51-1b0c-4867-bf22-4040cc08a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:29.493773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a4a51-1b0c-4867-bf22-4040cc08a0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:29.514426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-923a4a51-1b0c-4867-bf22-4040cc08a0f5 None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:29.515618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-923a4a51-1b0c-4867-bf22-4040cc08a0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:29.526588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 759/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Aug 12 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:29.534400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c547a7-e98a-4adc-b751-c43fd23e3aff None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:29.548984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c547a7-e98a-4adc-b751-c43fd23e3aff None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:29.554205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26361575-2a82-4f64-b2dc-a20621988d7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:29.554782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26361575-2a82-4f64-b2dc-a20621988d7a service placement] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:29.555030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26361575-2a82-4f64-b2dc-a20621988d7a service placement] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:29.555312 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26361575-2a82-4f64-b2dc-a20621988d7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:29.563425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26361575-2a82-4f64-b2dc-a20621988d7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:29.565029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26361575-2a82-4f64-b2dc-a20621988d7a service placement] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:29.574495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 762/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:48:31.928448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d1a25-a124-4e36-94c7-5d7230722614 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:31.929336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d1a25-a124-4e36-94c7-5d7230722614 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:31.929336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d1a25-a124-4e36-94c7-5d7230722614 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:31.930722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:31.935651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbb6b5-6ece-42a7-aac5-29e25315ef61 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:31.936113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbb6b5-6ece-42a7-aac5-29e25315ef61 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:31.936227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbb6b5-6ece-42a7-aac5-29e25315ef61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:31.957084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d9cbb6b5-6ece-42a7-aac5-29e25315ef61 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:31.958265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d9cbb6b5-6ece-42a7-aac5-29e25315ef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:31.967602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:32.129576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d1a25-a124-4e36-94c7-5d7230722614 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:32.155362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d1a25-a124-4e36-94c7-5d7230722614 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:32.165722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9405b-8b4d-4e34-9e43-c165eb704bf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:32.166316 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9405b-8b4d-4e34-9e43-c165eb704bf3 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:32.166586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9405b-8b4d-4e34-9e43-c165eb704bf3 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:32.166853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9405b-8b4d-4e34-9e43-c165eb704bf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:32.176309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9405b-8b4d-4e34-9e43-c165eb704bf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:32.177902 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9405b-8b4d-4e34-9e43-c165eb704bf3 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:32.188745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 761/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:48:33.000186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8cbc4-776c-46dd-804e-e08ab537596c None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:33.000833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8cbc4-776c-46dd-804e-e08ab537596c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:33.000833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8cbc4-776c-46dd-804e-e08ab537596c None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:33.002885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 10:48:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:33.011622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357dbd4-23eb-4877-9219-103c3c29a8e9 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:33.012038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357dbd4-23eb-4877-9219-103c3c29a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:33.012212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357dbd4-23eb-4877-9219-103c3c29a8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:33.043313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2357dbd4-23eb-4877-9219-103c3c29a8e9 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:33.045174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2357dbd4-23eb-4877-9219-103c3c29a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:33.058148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 10:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:33.212902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8cbc4-776c-46dd-804e-e08ab537596c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:33.235332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8cbc4-776c-46dd-804e-e08ab537596c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:33.241803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea1093-9489-468e-8baa-a1f038dde8b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:33.242401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea1093-9489-468e-8baa-a1f038dde8b6 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:33.242669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea1093-9489-468e-8baa-a1f038dde8b6 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:33.242929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea1093-9489-468e-8baa-a1f038dde8b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:33.251573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea1093-9489-468e-8baa-a1f038dde8b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:33.253131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea1093-9489-468e-8baa-a1f038dde8b6 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:33.263370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 765/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:48:37.149439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3935412-91e2-4d55-92b5-46613fcb7cae None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:37.150419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3935412-91e2-4d55-92b5-46613fcb7cae None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:37.151066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3935412-91e2-4d55-92b5-46613fcb7cae None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:37.152439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:48:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:37.158062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b878e1d-32d8-40b6-8e98-ccbd02a18752 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:37.158564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b878e1d-32d8-40b6-8e98-ccbd02a18752 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:37.158564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b878e1d-32d8-40b6-8e98-ccbd02a18752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:37.178828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6b878e1d-32d8-40b6-8e98-ccbd02a18752 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:37.180019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6b878e1d-32d8-40b6-8e98-ccbd02a18752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:37.188824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:48:37.471253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3935412-91e2-4d55-92b5-46613fcb7cae None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:37.473374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e3935412-91e2-4d55-92b5-46613fcb7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:37.509048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:48:37.554589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3935412-91e2-4d55-92b5-46613fcb7cae None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:37.558412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80b55f-86e2-4718-b10a-57b34eb38a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:37.559034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80b55f-86e2-4718-b10a-57b34eb38a40 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:37.559298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80b55f-86e2-4718-b10a-57b34eb38a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:37.559562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80b55f-86e2-4718-b10a-57b34eb38a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:37.569214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80b55f-86e2-4718-b10a-57b34eb38a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:37.570692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80b55f-86e2-4718-b10a-57b34eb38a40 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:37.583692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 764/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:48:39.468439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925ef0f-07b8-40a4-b211-a59c8329f187 None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:39.469122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925ef0f-07b8-40a4-b211-a59c8329f187 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:39.469122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925ef0f-07b8-40a4-b211-a59c8329f187 None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:39.470324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:39.475708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350975-7b47-4155-8678-135e85b15cea None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:39.476170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350975-7b47-4155-8678-135e85b15cea None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:39.476294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350975-7b47-4155-8678-135e85b15cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:39.496237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5b350975-7b47-4155-8678-135e85b15cea None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:39.497476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5b350975-7b47-4155-8678-135e85b15cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:39.506207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:39.788546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925ef0f-07b8-40a4-b211-a59c8329f187 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:39.800162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925ef0f-07b8-40a4-b211-a59c8329f187 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:39.804079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2ad25-25cd-418a-bf08-ee898302e325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:39.804699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2ad25-25cd-418a-bf08-ee898302e325 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:39.804971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2ad25-25cd-418a-bf08-ee898302e325 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:39.805241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2ad25-25cd-418a-bf08-ee898302e325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:39.815163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2ad25-25cd-418a-bf08-ee898302e325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:39.816654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2ad25-25cd-418a-bf08-ee898302e325 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:39.827068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 768/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:48:43.253264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fed9d-20e8-4b61-bf9c-7b08ba5f9eb3 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:43.253854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fed9d-20e8-4b61-bf9c-7b08ba5f9eb3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:43.253854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fed9d-20e8-4b61-bf9c-7b08ba5f9eb3 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:43.255115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:43.261440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3563b3-be91-4068-b1a3-d3508d53672b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:43.261904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3563b3-be91-4068-b1a3-d3508d53672b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:43.262012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3563b3-be91-4068-b1a3-d3508d53672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:43.287142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-9f3563b3-be91-4068-b1a3-d3508d53672b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:43.288383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9f3563b3-be91-4068-b1a3-d3508d53672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:43.296415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:48:43.840849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fed9d-20e8-4b61-bf9c-7b08ba5f9eb3 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:43.843530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-276fed9d-20e8-4b61-bf9c-7b08ba5f9eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:43.925862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fed9d-20e8-4b61-bf9c-7b08ba5f9eb3 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:43.929816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e00831-285d-4eb3-9bad-5ac3147c1b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:43.930407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00831-285d-4eb3-9bad-5ac3147c1b8d service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:43.930675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00831-285d-4eb3-9bad-5ac3147c1b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:43.930933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00831-285d-4eb3-9bad-5ac3147c1b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:43.940022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e00831-285d-4eb3-9bad-5ac3147c1b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:43.941474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e00831-285d-4eb3-9bad-5ac3147c1b8d service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:43.950729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 767/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:48:45.918358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc17d-979b-433b-807a-d0b7e6077bcd None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:45.918946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc17d-979b-433b-807a-d0b7e6077bcd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:45.918946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc17d-979b-433b-807a-d0b7e6077bcd None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:45.921003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 10:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:48:45.929259 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94640-7047-48a7-beb1-1929e479265e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:45.929722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94640-7047-48a7-beb1-1929e479265e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:45.929835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd94640-7047-48a7-beb1-1929e479265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:45.954334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2cd94640-7047-48a7-beb1-1929e479265e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:48:45.955536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2cd94640-7047-48a7-beb1-1929e479265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:45.965015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:48:46.510461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabc17d-979b-433b-807a-d0b7e6077bcd None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:48:46.513670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-eaabc17d-979b-433b-807a-d0b7e6077bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:48:46.537698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabc17d-979b-433b-807a-d0b7e6077bcd None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:48:46.543396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6eebdb-39a2-4051-aa3b-b24b1dae38c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:48:46.543989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eebdb-39a2-4051-aa3b-b24b1dae38c7 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:48:46.544273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eebdb-39a2-4051-aa3b-b24b1dae38c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:48:46.544558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eebdb-39a2-4051-aa3b-b24b1dae38c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:48:46.554516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eebdb-39a2-4051-aa3b-b24b1dae38c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:48:46.555954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eebdb-39a2-4051-aa3b-b24b1dae38c7 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:48:46.566486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 771/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:48:55.949385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=63 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:48:56.561981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=37 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:09.204870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f34a-afc7-449c-a206-cc8cb2729c1b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.212200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f34a-afc7-449c-a206-cc8cb2729c1b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.212200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6f34a-afc7-449c-a206-cc8cb2729c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.239315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3de6f34a-afc7-449c-a206-cc8cb2729c1b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.240618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3de6f34a-afc7-449c-a206-cc8cb2729c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.258099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 769/1540] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:09.267790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219a00d-8a9a-4645-8033-12ccd74aac5a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.268894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219a00d-8a9a-4645-8033-12ccd74aac5a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.269395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219a00d-8a9a-4645-8033-12ccd74aac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.287882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72da70-3a00-4336-a80d-b5ecf427f022 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.288229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72da70-3a00-4336-a80d-b5ecf427f022 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.288477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72da70-3a00-4336-a80d-b5ecf427f022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.296428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2219a00d-8a9a-4645-8033-12ccd74aac5a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.298183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2219a00d-8a9a-4645-8033-12ccd74aac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.314277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 772/1541] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.318964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219a00d-8a9a-4645-8033-12ccd74aac5a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.320355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9f72da70-3a00-4336-a80d-b5ecf427f022 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.321710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9f72da70-3a00-4336-a80d-b5ecf427f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.331328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219a00d-8a9a-4645-8033-12ccd74aac5a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.335111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 770/1542] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:09.343848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.344660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.348431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.348431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.348431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:09.348431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.348431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING py.warnings [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:50:09.348431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: warnings.warn(msg) Aug 12 10:50:09.348431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: Aug 12 10:50:09.349261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.357891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fafa4-7545-484f-b160-56eb6c7b10b3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.358270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fafa4-7545-484f-b160-56eb6c7b10b3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.358448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fafa4-7545-484f-b160-56eb6c7b10b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.373168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 773/1543] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:09.388622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.396922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-646fafa4-7545-484f-b160-56eb6c7b10b3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.398146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-646fafa4-7545-484f-b160-56eb6c7b10b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.403851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc34b58-c292-418e-8e36-32b89b7870dd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.411009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 771/1544] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.412252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4a25-44e4-4d4e-8759-1364ea7abb5b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.412651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4a25-44e4-4d4e-8759-1364ea7abb5b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.412924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4a25-44e4-4d4e-8759-1364ea7abb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.417875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d137e-41dd-4b47-afa5-1ae1ec37e32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.418495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d137e-41dd-4b47-afa5-1ae1ec37e32c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.418768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d137e-41dd-4b47-afa5-1ae1ec37e32c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.419038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d137e-41dd-4b47-afa5-1ae1ec37e32c admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.419720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d137e-41dd-4b47-afa5-1ae1ec37e32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.421708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d137e-41dd-4b47-afa5-1ae1ec37e32c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.441645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-714c4a25-44e4-4d4e-8759-1364ea7abb5b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.442855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-714c4a25-44e4-4d4e-8759-1364ea7abb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.458777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 772/1545] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.459573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4a25-44e4-4d4e-8759-1364ea7abb5b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.477800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 774/1546] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:09.479290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27f01-574c-404f-a1b1-eb29ee8bbb63 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.479416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27f01-574c-404f-a1b1-eb29ee8bbb63 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.479789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27f01-574c-404f-a1b1-eb29ee8bbb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.488306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4a25-44e4-4d4e-8759-1364ea7abb5b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.520970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-84f27f01-574c-404f-a1b1-eb29ee8bbb63 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.522282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-84f27f01-574c-404f-a1b1-eb29ee8bbb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.624075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2be0f-9fc7-4599-92b6-ee209a0b3d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.625178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2be0f-9fc7-4599-92b6-ee209a0b3d28 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.625640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2be0f-9fc7-4599-92b6-ee209a0b3d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.626103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2be0f-9fc7-4599-92b6-ee209a0b3d28 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.628123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2be0f-9fc7-4599-92b6-ee209a0b3d28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:09.628658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2be0f-9fc7-4599-92b6-ee209a0b3d28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.631164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2be0f-9fc7-4599-92b6-ee209a0b3d28 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.647841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 775/1547] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.649055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f27f01-574c-404f-a1b1-eb29ee8bbb63 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.651488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 773/1548] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:09.656358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c07c8-07fe-4f94-b7a4-62312b6d57e8 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.656358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c07c8-07fe-4f94-b7a4-62312b6d57e8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.656358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c07c8-07fe-4f94-b7a4-62312b6d57e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.664892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f27f01-574c-404f-a1b1-eb29ee8bbb63 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.669513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e2b31-c6c4-490a-9cff-8606b53f2d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.670113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e2b31-c6c4-490a-9cff-8606b53f2d86 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.670387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e2b31-c6c4-490a-9cff-8606b53f2d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.670649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e2b31-c6c4-490a-9cff-8606b53f2d86 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.671265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e2b31-c6c4-490a-9cff-8606b53f2d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.672311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING py.warnings [None req-118e2b31-c6c4-490a-9cff-8606b53f2d86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:50:09.672311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: warnings.warn(msg) Aug 12 10:50:09.672311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: Aug 12 10:50:09.673713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e2b31-c6c4-490a-9cff-8606b53f2d86 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.675415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5c7c07c8-07fe-4f94-b7a4-62312b6d57e8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.677036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5c7c07c8-07fe-4f94-b7a4-62312b6d57e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.685944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 774/1549] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.687280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc165b5-ea99-4019-9380-98df1cd73489 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.687606 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc165b5-ea99-4019-9380-98df1cd73489 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.687946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc165b5-ea99-4019-9380-98df1cd73489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.701565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 776/1550] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:09.701565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607001b4-3c74-4d0e-8353-22ab1bc85adc None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.701565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607001b4-3c74-4d0e-8353-22ab1bc85adc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.701565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607001b4-3c74-4d0e-8353-22ab1bc85adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.708547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8dc165b5-ea99-4019-9380-98df1cd73489 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.709807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8dc165b5-ea99-4019-9380-98df1cd73489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.718278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 775/1551] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.719820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc165b5-ea99-4019-9380-98df1cd73489 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.722802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-607001b4-3c74-4d0e-8353-22ab1bc85adc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.724219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-607001b4-3c74-4d0e-8353-22ab1bc85adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.731601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc165b5-ea99-4019-9380-98df1cd73489 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.732954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 777/1552] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.734225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04fe6-2204-4cd7-a6ab-a9ee181fc4c2 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.734497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04fe6-2204-4cd7-a6ab-a9ee181fc4c2 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.734764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04fe6-2204-4cd7-a6ab-a9ee181fc4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.740344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a10a1-ab98-47eb-943f-4365125702d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.741021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a10a1-ab98-47eb-943f-4365125702d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.741291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a10a1-ab98-47eb-943f-4365125702d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.741572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a10a1-ab98-47eb-943f-4365125702d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.742214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a10a1-ab98-47eb-943f-4365125702d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:09.742508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a10a1-ab98-47eb-943f-4365125702d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.744099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a10a1-ab98-47eb-943f-4365125702d9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.754511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-49c04fe6-2204-4cd7-a6ab-a9ee181fc4c2 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.755728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-49c04fe6-2204-4cd7-a6ab-a9ee181fc4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.763412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 776/1553] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:09.763412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0ad3c-1aaf-4a46-a929-ac4297c521a4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.763412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0ad3c-1aaf-4a46-a929-ac4297c521a4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.763412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0ad3c-1aaf-4a46-a929-ac4297c521a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.771268 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 778/1554] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.772104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c04fe6-2204-4cd7-a6ab-a9ee181fc4c2 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.781738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c04fe6-2204-4cd7-a6ab-a9ee181fc4c2 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.782164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-32b0ad3c-1aaf-4a46-a929-ac4297c521a4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.783641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-32b0ad3c-1aaf-4a46-a929-ac4297c521a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.787182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d0897-45a5-4238-815c-640fbfa29745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.787902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0897-45a5-4238-815c-640fbfa29745 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.788238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0897-45a5-4238-815c-640fbfa29745 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.788523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0897-45a5-4238-815c-640fbfa29745 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.789103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d0897-45a5-4238-815c-640fbfa29745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.791193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d0897-45a5-4238-815c-640fbfa29745 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.791672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 777/1555] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.792940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2f9b1-e2ea-4f12-b2cf-8c54b904682f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.793219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2f9b1-e2ea-4f12-b2cf-8c54b904682f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.793519 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2f9b1-e2ea-4f12-b2cf-8c54b904682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.812133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e7a2f9b1-e2ea-4f12-b2cf-8c54b904682f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.813488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e7a2f9b1-e2ea-4f12-b2cf-8c54b904682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.821577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 778/1556] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.822423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2f9b1-e2ea-4f12-b2cf-8c54b904682f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.826629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 779/1557] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:09.826629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3a89-bf82-4691-8eaf-52a717b971cd None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.826629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3a89-bf82-4691-8eaf-52a717b971cd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.826629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc3a89-bf82-4691-8eaf-52a717b971cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.832196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2f9b1-e2ea-4f12-b2cf-8c54b904682f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.847996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2ccc3a89-bf82-4691-8eaf-52a717b971cd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.849341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2ccc3a89-bf82-4691-8eaf-52a717b971cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.887952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25b9f9-ab3d-42d9-a80b-0e843e034340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.888585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25b9f9-ab3d-42d9-a80b-0e843e034340 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.888858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25b9f9-ab3d-42d9-a80b-0e843e034340 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.889121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25b9f9-ab3d-42d9-a80b-0e843e034340 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.889753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25b9f9-ab3d-42d9-a80b-0e843e034340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.891353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25b9f9-ab3d-42d9-a80b-0e843e034340 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.900757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:09.902675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 779/1558] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:09.904070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f3e-1d2d-47b3-875c-3c83784a8448 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.904368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f3e-1d2d-47b3-875c-3c83784a8448 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.904715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f3e-1d2d-47b3-875c-3c83784a8448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.912291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 780/1559] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.913406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc3a89-bf82-4691-8eaf-52a717b971cd None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:09.925178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc3a89-bf82-4691-8eaf-52a717b971cd None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:09.927524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-740a5f3e-1d2d-47b3-875c-3c83784a8448 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.928741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-740a5f3e-1d2d-47b3-875c-3c83784a8448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.930347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaee14a-39c1-4ab0-ad5b-4ab550526233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:09.930939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee14a-39c1-4ab0-ad5b-4ab550526233 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.931216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee14a-39c1-4ab0-ad5b-4ab550526233 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.931499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee14a-39c1-4ab0-ad5b-4ab550526233 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.932085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaee14a-39c1-4ab0-ad5b-4ab550526233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:09.934119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaee14a-39c1-4ab0-ad5b-4ab550526233 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:09.937373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 780/1560] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:09.938609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244ad0-7829-436d-834a-f2e8c93badeb None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.938908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244ad0-7829-436d-834a-f2e8c93badeb None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.939293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23244ad0-7829-436d-834a-f2e8c93badeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.959222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-23244ad0-7829-436d-834a-f2e8c93badeb None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.960509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-23244ad0-7829-436d-834a-f2e8c93badeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:09.965134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 781/1561] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:09.965134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a686bdb-43d1-40e0-814b-cb18b1e92594 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:09.965134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a686bdb-43d1-40e0-814b-cb18b1e92594 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:09.965134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a686bdb-43d1-40e0-814b-cb18b1e92594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:09.974298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:09.984997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4a686bdb-43d1-40e0-814b-cb18b1e92594 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:09.986323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4a686bdb-43d1-40e0-814b-cb18b1e92594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:10.024600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 781/1562] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:10.025776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244ad0-7829-436d-834a-f2e8c93badeb None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.035728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23244ad0-7829-436d-834a-f2e8c93badeb None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.040335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.040922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.041193 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.041476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.042136 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:10.042429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.044015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.045977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 782/1563] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:10.049765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a686bdb-43d1-40e0-814b-cb18b1e92594 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.055912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a686bdb-43d1-40e0-814b-cb18b1e92594 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.059547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 782/1564] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:10.060388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.060647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.061277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.061504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.061770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.062389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.064397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.070309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdd8d9-0d27-403c-83bc-50f8b951cf1d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.075032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.075636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.075937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.076245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.076987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.078889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.087344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 783/1565] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.088230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.090092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 783/1566] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.090775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.098757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0218a27-65c7-471b-b95b-070aa24643c9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.100910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1ac8e-e1d9-419d-a65e-4976fe9538e7 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.103447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.104038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.104336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.104607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.105283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:10.105460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.105584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.105969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.106239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.106509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.107128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:10.107234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.107417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.109013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.119350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 784/1567] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:10.120134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.127846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 784/1568] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:10.127846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.137346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df0dc9-214e-4dd4-b196-8235849fec38 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.137542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a523a5a-378b-4462-9a9b-904f5bab5b17 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.142007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.142182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.142968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.143118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.143240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.143377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.143501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.143611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.144137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:10.144297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.144486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.146044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.146171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.167191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 785/1569] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:10.167191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.168216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 785/1570] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.169045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.171633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb2d12-1ba6-4f0c-a767-a09a6373eb1b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.176797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.177357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.177677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.177944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.178562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:10.178854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.180461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.181001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524017c-8678-43e0-b7bc-a8931cdaec36 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.185945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.186704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.186978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.187755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.188411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.190506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.192154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 786/1571] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:10.193067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.204589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0acb-bbb3-4ac4-a4c4-45dbe735b905 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.209316 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.210043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.210310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.210646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] PATH_INFO: `/v3/projects/18fbf543ce304e6c8d7c758760d6c08d/users/00c02d5ccb7d44dbab9d2846df39b025/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.223542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 786/1572] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.223542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.226030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=18fbf543ce304e6c8d7c758760d6c08d, user_id=00c02d5ccb7d44dbab9d2846df39b025, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.227063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:50:10.227063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:50:10.227063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:50:10.229038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.243075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 787/1573] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:10 2022] PUT /identity/v3/projects/18fbf543ce304e6c8d7c758760d6c08d/users/00c02d5ccb7d44dbab9d2846df39b025/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:10.254560 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.287863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160d2e2-2a03-4ab2-8023-be9594c0024b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.297859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:10.307712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3f821-14dd-4feb-9b0f-58537c1763ee admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.312758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.313571 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.314129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.314491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.315126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.316374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:50:10.316374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:50:10.316374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:50:10.317668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.347042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 788/1574] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.348056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.357012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.357748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.358022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.358348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.358926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.360569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.368459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:10.370853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 787/1575] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.371646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.381754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d152-78db-4524-8ce7-6f06293af576 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.415260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512a0c-a24a-4e70-89f1-b9c9d434b39e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.420178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.420833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.421104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.421485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.422125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.424182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.437855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.438555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.439105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.439770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.440513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.442774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.452636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 789/1576] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.456698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.471862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 788/1577] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.473025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.519256 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7aa121-721b-4604-a8c1-18d4e5505f2b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.533421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e1c88-4d51-4318-9738-806c6e3109d6 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.538133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.538703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.538970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.539232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.539857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.541984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.570223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 789/1578] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.574662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.577073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a63f0-ca88-42ec-ab30-c8ec7e694081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.577730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a63f0-ca88-42ec-ab30-c8ec7e694081 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.578021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a63f0-ca88-42ec-ab30-c8ec7e694081 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.578267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a63f0-ca88-42ec-ab30-c8ec7e694081 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.578914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a63f0-ca88-42ec-ab30-c8ec7e694081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.580963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a63f0-ca88-42ec-ab30-c8ec7e694081 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.593995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 790/1579] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.593995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872ddf-4b8e-49d1-a387-5d2c90a9db19 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.593995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872ddf-4b8e-49d1-a387-5d2c90a9db19 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.593995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872ddf-4b8e-49d1-a387-5d2c90a9db19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.596749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:10.622339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-25872ddf-4b8e-49d1-a387-5d2c90a9db19 None None] MFA Rules not processed for user `00c02d5ccb7d44dbab9d2846df39b025`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:10.623552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-25872ddf-4b8e-49d1-a387-5d2c90a9db19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:10.640028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ad6ed-17db-47ce-b5bc-82d45e127921 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.645869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.646472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.646744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.647011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.647632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.649584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.659281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:10.672729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 790/1580] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.673897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.683366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281522ed-901a-4c91-9960-20a8f1ec0a05 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.687860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.688464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.688735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.689063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.689705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.691714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.695320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 791/1581] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:10.696343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25872ddf-4b8e-49d1-a387-5d2c90a9db19 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.706738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25872ddf-4b8e-49d1-a387-5d2c90a9db19 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.711468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.712094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.712403 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.712674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.713314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.713440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 791/1582] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.714292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.715038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.724901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4ef07-0698-41dc-b3ff-3143085736f5 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.729986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 792/1583] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.729986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.733147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.733786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.734062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.734443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.734953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.737284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.740204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaff4a-bd4b-4a4e-a804-08d77f4ff3e9 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.744789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.745375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.745649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.745918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.746530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.748490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.761214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 792/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/users => generated 296 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.762325 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.773230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 793/1585] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.773505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050bbc76-d206-49f7-9527-7e2442e4fbdf admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.773914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.778240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.778888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.779161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.779425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.780041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.783050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.784105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c277b-aa33-4e69-b5ce-39979d991c97 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.788572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.789182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.789452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.789713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.790336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.791907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.803962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 794/1586] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.804891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.805788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 793/1587] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.806576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.815572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666769c-0af7-49b5-b495-4898d03d3c2b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.816534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7eed7-d823-4dc0-9b18-221c05867569 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.820174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.820795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.820958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.821079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.821320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.821532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.821801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.821950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.822065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.822668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.823533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.824555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.833531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 795/1588] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.834322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.847214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe046a-a423-42b0-b35d-480a472459ad admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.852899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.853889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.854161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.854432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.855057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.856680 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.860241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 794/1589] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:10 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:10.861959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.867140 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 796/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.867946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.877298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc242f0-bd05-4d27-a9f9-a0c6864f9452 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.924738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b56669-7602-4f35-a6c4-30b423a7c26d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.929255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.929905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.930176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.930440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] PATH_INFO: `/v3/projects/09c9254199eb4abd9cd1f158e8485935/users/39e66fc5b67f4c17af43b6d1fe134dc4/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.933967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:10.934586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:10.934866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:10.935137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:10.935775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.937440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.944874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c9254199eb4abd9cd1f158e8485935, user_id=39e66fc5b67f4c17af43b6d1fe134dc4, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:10.947597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:10.948627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 797/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:10.952940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.962755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 795/1592] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:10 2022] PUT /identity/v3/projects/09c9254199eb4abd9cd1f158e8485935/users/39e66fc5b67f4c17af43b6d1fe134dc4/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:10.963642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:10.963746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822406e-6d2b-4764-b830-007388d4b506 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:10.983943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:11.027920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.028559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.028830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.029093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.029226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9331af-9431-4323-a11b-3998ec3226a9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.029727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.031314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.033930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.034526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.034796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.035063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.035697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.037352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.042067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 798/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.046004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.047695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 796/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.048605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.052927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:11.057317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29d694-befb-42de-b72b-9954e2723043 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.058040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d45356-75eb-4bca-bbec-007e2710861a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.063101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.063411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.063750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.064062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.064756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.064857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.065015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.065187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.065516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] PATH_INFO: `/v3/projects/e778a4db38cd407092cb639b9738e35a/users/d49986e88ae049bc8146010dd3a79af2/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.066681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.077038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 797/1595] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.080919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e778a4db38cd407092cb639b9738e35a, user_id=d49986e88ae049bc8146010dd3a79af2, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.081232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.083505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.091959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14b132-19a0-42c6-9639-0a95aedd6260 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.098547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 799/1596] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:11 2022] PUT /identity/v3/projects/e778a4db38cd407092cb639b9738e35a/users/d49986e88ae049bc8146010dd3a79af2/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:11.099349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.162371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb398a-fd8b-41af-a831-ee0eb3e07494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.162972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb398a-fd8b-41af-a831-ee0eb3e07494 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.163241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb398a-fd8b-41af-a831-ee0eb3e07494 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.163488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb398a-fd8b-41af-a831-ee0eb3e07494 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.163628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d8d07-6f80-4132-bf88-a41169997d0f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.164162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb398a-fd8b-41af-a831-ee0eb3e07494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.166072 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb398a-fd8b-41af-a831-ee0eb3e07494 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.168217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.168853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.169152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.169421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.170057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.171682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.181814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 800/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.182804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.191751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 798/1598] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:11.192224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd449f-9b4b-4f11-99b3-2253c5c863d7 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.192895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213c638-51fd-4602-ba8e-151d70d0c0eb None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.193171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213c638-51fd-4602-ba8e-151d70d0c0eb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.193441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213c638-51fd-4602-ba8e-151d70d0c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.197515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.198109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.198384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.198653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.199287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.200925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.211125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 801/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.215100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.221831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604fc75-ec13-4e1c-b89a-1c5a48a63adc admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.224651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8213c638-51fd-4602-ba8e-151d70d0c0eb None None] MFA Rules not processed for user `39e66fc5b67f4c17af43b6d1fe134dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.225944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8213c638-51fd-4602-ba8e-151d70d0c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.226377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.226963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.227253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.228552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.229201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.230794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.242413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 802/1600] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.243165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.253697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cdef2-94d7-46fd-bd9a-644adc56ad35 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.258386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25235214-b437-4644-936d-051426680183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.259429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25235214-b437-4644-936d-051426680183 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.259700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25235214-b437-4644-936d-051426680183 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.259965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25235214-b437-4644-936d-051426680183 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.260648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25235214-b437-4644-936d-051426680183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.262230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25235214-b437-4644-936d-051426680183 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.273661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 803/1601] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.274558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25235214-b437-4644-936d-051426680183 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.283801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25235214-b437-4644-936d-051426680183 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.288365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.288986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.289257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.289522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] PATH_INFO: `/v3/projects/362a04e0181b4fa49e27fb38c04ac677/users/e0d5ff60d4b24fe0bbd771e8ed97db2f/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.295401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 799/1602] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.296531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a14f54-64c1-4c70-9619-ab679fa6a09b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.296810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a14f54-64c1-4c70-9619-ab679fa6a09b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.297074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a14f54-64c1-4c70-9619-ab679fa6a09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.307078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=362a04e0181b4fa49e27fb38c04ac677, user_id=e0d5ff60d4b24fe0bbd771e8ed97db2f, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.309806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.326767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-75a14f54-64c1-4c70-9619-ab679fa6a09b None None] MFA Rules not processed for user `d49986e88ae049bc8146010dd3a79af2`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.327512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 804/1603] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:11 2022] PUT /identity/v3/projects/362a04e0181b4fa49e27fb38c04ac677/users/e0d5ff60d4b24fe0bbd771e8ed97db2f/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:11.328695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.331673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-75a14f54-64c1-4c70-9619-ab679fa6a09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.345416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:11.399440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920e519-cde1-480c-a059-43064841f460 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.403359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 800/1604] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.404195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a14f54-64c1-4c70-9619-ab679fa6a09b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.404340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.404948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.405220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.405559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] PATH_INFO: `/v3/projects/1d3cc44ffed94a428ab9e49ebb3854e0/users/c0711a144b4c4432ab849367d3bd2b33/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.418030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a14f54-64c1-4c70-9619-ab679fa6a09b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.421478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3cc44ffed94a428ab9e49ebb3854e0, user_id=c0711a144b4c4432ab849367d3bd2b33, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.422806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.423589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.423919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.424204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.424371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.425101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.426792 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.441934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 801/1605] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.441934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.442434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 805/1606] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:11 2022] PUT /identity/v3/projects/1d3cc44ffed94a428ab9e49ebb3854e0/users/c0711a144b4c4432ab849367d3bd2b33/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:11.442434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.449875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:11.453047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9524-5dc5-4e97-8735-deeb0f1c39f5 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.510172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.510746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.511003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb6fc0-66b2-4373-b65f-e9ecc6627774 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.511108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.511282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] PATH_INFO: `/v3/projects/dc63f648a390465e94ab8170362a9b01/users/58e209063f5c454d87d9f743bedfc734/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.515570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954c6aa-f2d7-4cac-a4bc-7c3ffa4501ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.516229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c6aa-f2d7-4cac-a4bc-7c3ffa4501ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.516531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c6aa-f2d7-4cac-a4bc-7c3ffa4501ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.516795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c6aa-f2d7-4cac-a4bc-7c3ffa4501ca admin admin] PATH_INFO: `/v3/projects/dd9ae370235744339aedbbbe26570b22/users/2bdd2c709e394c57a8d5cd221db552f9/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.526128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc63f648a390465e94ab8170362a9b01, user_id=58e209063f5c454d87d9f743bedfc734, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.528841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.535188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954c6aa-f2d7-4cac-a4bc-7c3ffa4501ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9ae370235744339aedbbbe26570b22, user_id=2bdd2c709e394c57a8d5cd221db552f9, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.538106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954c6aa-f2d7-4cac-a4bc-7c3ffa4501ca admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.543557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 802/1607] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:11 2022] PUT /identity/v3/projects/dc63f648a390465e94ab8170362a9b01/users/58e209063f5c454d87d9f743bedfc734/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:11.544535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.556384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 806/1608] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:11 2022] PUT /identity/v3/projects/dd9ae370235744339aedbbbe26570b22/users/2bdd2c709e394c57a8d5cd221db552f9/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:11.558018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112aa98-0658-4838-a0fe-0ad793334f59 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.558153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112aa98-0658-4838-a0fe-0ad793334f59 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.558438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112aa98-0658-4838-a0fe-0ad793334f59 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.559906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 807/1609] 10.222.0.23 () {54 vars in 962 bytes} [Fri Aug 12 10:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:11.560918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1f786-1138-406c-b3ef-e678dd6d6c36 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.561329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1f786-1138-406c-b3ef-e678dd6d6c36 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.561653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1f786-1138-406c-b3ef-e678dd6d6c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.582098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c2a1f786-1138-406c-b3ef-e678dd6d6c36 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.583281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c2a1f786-1138-406c-b3ef-e678dd6d6c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.610904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a13089-6985-4f03-ad80-f321b0906d49 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.639857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:11.640931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 808/1610] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.642785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e679-22d4-4996-ad4e-dd75e33d4b97 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.643069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e679-22d4-4996-ad4e-dd75e33d4b97 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.643340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e679-22d4-4996-ad4e-dd75e33d4b97 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.644939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 809/1611] 10.222.0.23 () {54 vars in 962 bytes} [Fri Aug 12 10:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:11.646002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336e679-22d4-4996-ad4e-dd75e33d4b97 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.663054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336e679-22d4-4996-ad4e-dd75e33d4b97 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.667762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362364a-45ff-46c0-9737-84132cb97015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.668495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362364a-45ff-46c0-9737-84132cb97015 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.668780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362364a-45ff-46c0-9737-84132cb97015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.669050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362364a-45ff-46c0-9737-84132cb97015 admin admin] PATH_INFO: `/v3/projects/0f26dfd61cb242d682b6e42552fb2786/users/7fa3b07d377f49ee8f01f425efbf6576/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.670740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab8d4d-41c8-4460-80a9-8560a532b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.671397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab8d4d-41c8-4460-80a9-8560a532b3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.671668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab8d4d-41c8-4460-80a9-8560a532b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.671941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab8d4d-41c8-4460-80a9-8560a532b3d8 admin admin] PATH_INFO: `/v3/projects/9475bf21c0234ae5bf55a9c2ba967d33/users/5c7c0993b20843e4804fa96cd9be34d1/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.683753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362364a-45ff-46c0-9737-84132cb97015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f26dfd61cb242d682b6e42552fb2786, user_id=7fa3b07d377f49ee8f01f425efbf6576, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.686317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab8d4d-41c8-4460-80a9-8560a532b3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9475bf21c0234ae5bf55a9c2ba967d33, user_id=5c7c0993b20843e4804fa96cd9be34d1, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.686487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362364a-45ff-46c0-9737-84132cb97015 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.689092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab8d4d-41c8-4460-80a9-8560a532b3d8 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.701377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 810/1612] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:11 2022] PUT /identity/v3/projects/0f26dfd61cb242d682b6e42552fb2786/users/7fa3b07d377f49ee8f01f425efbf6576/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:11.702704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbaee7-a746-4994-b897-5befb01971c1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.703081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbaee7-a746-4994-b897-5befb01971c1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.703389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 803/1613] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:11 2022] PUT /identity/v3/projects/9475bf21c0234ae5bf55a9c2ba967d33/users/5c7c0993b20843e4804fa96cd9be34d1/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:11.703501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbaee7-a746-4994-b897-5befb01971c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.704706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ddd6a-417b-4fb2-a776-3148bdebf083 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.704985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ddd6a-417b-4fb2-a776-3148bdebf083 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.705253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ddd6a-417b-4fb2-a776-3148bdebf083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.723749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8d5ddd6a-417b-4fb2-a776-3148bdebf083 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.724978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8d5ddd6a-417b-4fb2-a776-3148bdebf083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.725241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ebfbaee7-a746-4994-b897-5befb01971c1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.726419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ebfbaee7-a746-4994-b897-5befb01971c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.784646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 804/1614] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.786218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209444e-11ff-4b5a-89d9-12a5a5b58cca None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.786642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209444e-11ff-4b5a-89d9-12a5a5b58cca None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.787021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 811/1615] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.787218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209444e-11ff-4b5a-89d9-12a5a5b58cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.788276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ce692-4c42-4221-9c2e-2a83a2ea411f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.789024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ce692-4c42-4221-9c2e-2a83a2ea411f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.789024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ce692-4c42-4221-9c2e-2a83a2ea411f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.797476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:11.817407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-085ce692-4c42-4221-9c2e-2a83a2ea411f None None] MFA Rules not processed for user `2bdd2c709e394c57a8d5cd221db552f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.818625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-085ce692-4c42-4221-9c2e-2a83a2ea411f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.825250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0209444e-11ff-4b5a-89d9-12a5a5b58cca None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.826413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0209444e-11ff-4b5a-89d9-12a5a5b58cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.883997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 812/1616] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.885446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bff9-83a5-4d9a-ab87-c874c3434926 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.885730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bff9-83a5-4d9a-ab87-c874c3434926 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.885999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bff9-83a5-4d9a-ab87-c874c3434926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.893596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 805/1617] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.894432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209444e-11ff-4b5a-89d9-12a5a5b58cca None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.906603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b537bff9-83a5-4d9a-ab87-c874c3434926 None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.908085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b537bff9-83a5-4d9a-ab87-c874c3434926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.917741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 813/1618] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.919090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e2357-ede2-42cc-a341-8280b3c65069 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.919367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e2357-ede2-42cc-a341-8280b3c65069 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.919636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e2357-ede2-42cc-a341-8280b3c65069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.931522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:11.937884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2c6e2357-ede2-42cc-a341-8280b3c65069 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:11.939165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2c6e2357-ede2-42cc-a341-8280b3c65069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.947363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 814/1619] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:11.948234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e2357-ede2-42cc-a341-8280b3c65069 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.957899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e2357-ede2-42cc-a341-8280b3c65069 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.958061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209444e-11ff-4b5a-89d9-12a5a5b58cca None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.961694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.962390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.962725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.963053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.963347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.963917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.964191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.964493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.965363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.966816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.973294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:11.974736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:11.977483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 815/1620] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:11.978290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.980192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:11.984722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 806/1621] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:11.985410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:11.992548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31561ac5-c4b8-4bd1-a822-86bf6cb7c825 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.993403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb40204-ef82-47cb-b4fb-9cdc67fc2801 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:11.996487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.997087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.997395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.997500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:11.997699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:11.998084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:11.998357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:11.998627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.008011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.008201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.009529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.009712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.021683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 807/1622] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:12.022461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.023985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 816/1623] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:12.024835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.035808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c47d98-afe2-464b-8522-2e66d2a28399 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.041215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.042057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.042319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.042584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.043255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.045745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.051055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20794-194b-4754-ab39-8a809237e08e service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.059354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.060067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.060447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.060789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.086243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 808/1624] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:12 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:12.087228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.154542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.156642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.170644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 817/1625] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:12.172029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.184794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a509d-d5f3-4240-8919-c4976a2503e3 service nova] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.197613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba8a2-fb88-4db6-9287-0cdb6ca20795 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.205155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.205924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.206470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.206965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.255461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.256784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.256784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.256984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.263780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:12.320509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.322398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.335776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.336006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 809/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:12.338380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.346530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.357463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d194c-15c8-4f8d-82fb-511a6b7163d4 service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.364943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8fa32-c0c0-42b2-9b6b-6d7ccac33dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.365641 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8fa32-c0c0-42b2-9b6b-6d7ccac33dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.366574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8fa32-c0c0-42b2-9b6b-6d7ccac33dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.366943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8fa32-c0c0-42b2-9b6b-6d7ccac33dc3 admin admin] PATH_INFO: `/v3/projects/0f26dfd61cb242d682b6e42552fb2786/users/7fa3b07d377f49ee8f01f425efbf6576/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.369484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8fa32-c0c0-42b2-9b6b-6d7ccac33dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f26dfd61cb242d682b6e42552fb2786, user_id=7fa3b07d377f49ee8f01f425efbf6576, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.373631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8fa32-c0c0-42b2-9b6b-6d7ccac33dc3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.385634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 818/1627] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:12.390436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 810/1628] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:12 2022] PUT /identity/v3/projects/0f26dfd61cb242d682b6e42552fb2786/users/7fa3b07d377f49ee8f01f425efbf6576/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:12.390583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.401802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b43872-d700-4e22-93fa-23200232367a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.402558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b43872-d700-4e22-93fa-23200232367a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.402944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b43872-d700-4e22-93fa-23200232367a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.418399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a7fd2-a536-44e7-bce2-f051aa7a7348 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.439629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-43b43872-d700-4e22-93fa-23200232367a None None] MFA Rules not processed for user `7fa3b07d377f49ee8f01f425efbf6576`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:12.441033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-43b43872-d700-4e22-93fa-23200232367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:12.458075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:12.539967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 811/1629] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:12.539967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585376-21d1-4d41-a88b-e81da9ee4dbf None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.539967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585376-21d1-4d41-a88b-e81da9ee4dbf None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.539967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585376-21d1-4d41-a88b-e81da9ee4dbf None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.541911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 812/1630] 10.222.0.23 () {54 vars in 912 bytes} [Fri Aug 12 10:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:12.546607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.547125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.547400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.547667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.548367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.551254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2af1f-5551-4c1a-b1ca-87f763657622 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.551420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.551677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2af1f-5551-4c1a-b1ca-87f763657622 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.552017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2af1f-5551-4c1a-b1ca-87f763657622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.580594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4ef2af1f-5551-4c1a-b1ca-87f763657622 None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:12.581784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4ef2af1f-5551-4c1a-b1ca-87f763657622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:12.587627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 819/1631] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:12 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:12.592393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.701069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 813/1632] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:50:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:12.702137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2af1f-5551-4c1a-b1ca-87f763657622 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.715905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2af1f-5551-4c1a-b1ca-87f763657622 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.724035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.724985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.725338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.725728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.726183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d455-9a27-441d-83e2-b3d3d5b0a80e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.726559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.728596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.730265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.730944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.731273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.731641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.743153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 814/1633] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:12.744200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.746859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.748876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.771175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5258d-a91d-4cc4-bc3c-beb581dd8ee9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.776032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.776758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.777037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.777886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.787221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 820/1634] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:12.791795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.815692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.818226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.820862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23712f-dd97-4b97-bd26-fcbfbcc6e20b service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.833432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d6c60-1644-4b4b-bf29-f5159ebd5275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.835261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d6c60-1644-4b4b-bf29-f5159ebd5275 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.836089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d6c60-1644-4b4b-bf29-f5159ebd5275 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.836393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d6c60-1644-4b4b-bf29-f5159ebd5275 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.838234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d6c60-1644-4b4b-bf29-f5159ebd5275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.843741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d6c60-1644-4b4b-bf29-f5159ebd5275 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:12.847551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 815/1635] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:12.879093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 821/1636] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:12.883369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:12.906658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44856-45a7-420b-871e-4c2e4411eed5 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:12.915035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013db517-425b-47b1-a4ff-18bf0596d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:12.915180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013db517-425b-47b1-a4ff-18bf0596d85b admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:12.915526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013db517-425b-47b1-a4ff-18bf0596d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:12.915652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013db517-425b-47b1-a4ff-18bf0596d85b admin admin] PATH_INFO: `/v3/projects/daef9c084bf04f749f314c3a22e39ec7/users/fe5641a8c7bd4d29945dbab81bfdd2ca/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:12.969893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013db517-425b-47b1-a4ff-18bf0596d85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=daef9c084bf04f749f314c3a22e39ec7, user_id=fe5641a8c7bd4d29945dbab81bfdd2ca, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:12.991638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013db517-425b-47b1-a4ff-18bf0596d85b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:13.012614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 816/1637] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:12 2022] PUT /identity/v3/projects/daef9c084bf04f749f314c3a22e39ec7/users/fe5641a8c7bd4d29945dbab81bfdd2ca/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:13.023202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd0a09-f4e5-49f6-ace3-4d2a5a29db1e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.024301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd0a09-f4e5-49f6-ace3-4d2a5a29db1e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.024301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd0a09-f4e5-49f6-ace3-4d2a5a29db1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.072964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a0c83-0ab3-43b3-880f-6de32daa66fa None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.073355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a0c83-0ab3-43b3-880f-6de32daa66fa None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.073730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a0c83-0ab3-43b3-880f-6de32daa66fa None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.075432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 817/1638] 10.222.0.23 () {54 vars in 912 bytes} [Fri Aug 12 10:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:50:13.082611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ecfd0a09-f4e5-49f6-ace3-4d2a5a29db1e None None] MFA Rules not processed for user `fe5641a8c7bd4d29945dbab81bfdd2ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:13.087208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c3d50-6b8d-4e1d-81fa-61f93990707c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.087594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c3d50-6b8d-4e1d-81fa-61f93990707c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.088119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c3d50-6b8d-4e1d-81fa-61f93990707c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.089369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ecfd0a09-f4e5-49f6-ace3-4d2a5a29db1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:13.122327 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:13.131000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-554c3d50-6b8d-4e1d-81fa-61f93990707c None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:13.132581 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-554c3d50-6b8d-4e1d-81fa-61f93990707c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:13.268914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:13.273279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 822/1639] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:13.278062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd0a09-f4e5-49f6-ace3-4d2a5a29db1e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:13.321814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 818/1640] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:50:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:13.375711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c3d50-6b8d-4e1d-81fa-61f93990707c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:13.426425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd0a09-f4e5-49f6-ace3-4d2a5a29db1e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:13.433552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:13.434547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.435025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.435451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.436713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:13.442921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:13.508142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c3d50-6b8d-4e1d-81fa-61f93990707c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:13.510533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 823/1641] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:13 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:13.516181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:13.613732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:13.614476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.614754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.615281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.661176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113d5f1-6603-4cb4-8a34-a55323929588 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:13.672514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:13.674785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.676207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.676795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.679471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:13.684132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:13.736020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:13.738126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:13.747232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 824/1642] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:13 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:13.754654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:13.782310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f929b1-8583-4670-b7b4-170d4b81c33f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:13.782793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 819/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:13.801432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:13.804649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.805696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.806047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.807738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:13.811839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:13.881161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 825/1644] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:13.884317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:13.907603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2802db-3a43-455c-a15f-1d3190a3cfed service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:13.930264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:13.933003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:13.933474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:13.934015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] PATH_INFO: `/v3/projects/5c5275f0853a431f9d13a9764fec8b30/users/1d77393c5a2b47f8811d189bfc67f82f/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:13.988403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5275f0853a431f9d13a9764fec8b30, user_id=1d77393c5a2b47f8811d189bfc67f82f, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:13.997763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:14.035802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 820/1645] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:13 2022] PUT /identity/v3/projects/5c5275f0853a431f9d13a9764fec8b30/users/1d77393c5a2b47f8811d189bfc67f82f/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:14.040569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:14.055523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9080-617a-4d6b-bde9-598e7c85f946 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:14.090498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:14.163484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:14.209059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:14.232139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:14.233154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.233579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.234100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:14.235596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:14.237977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:14.254080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 826/1646] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:14.267917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:14.283326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234352a6-9411-48e1-aeb2-f37f182ab02f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:14.289027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4204d42-0174-4235-9304-c909f87b9b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:14.289992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204d42-0174-4235-9304-c909f87b9b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.290153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204d42-0174-4235-9304-c909f87b9b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.290349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204d42-0174-4235-9304-c909f87b9b7f admin admin] PATH_INFO: `/v3/projects/5c5275f0853a431f9d13a9764fec8b30/users/1d77393c5a2b47f8811d189bfc67f82f/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:14.292384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4204d42-0174-4235-9304-c909f87b9b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5275f0853a431f9d13a9764fec8b30, user_id=1d77393c5a2b47f8811d189bfc67f82f, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:14.295173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4204d42-0174-4235-9304-c909f87b9b7f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:14.307064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa0818-af69-4f92-a658-99cdd8428cbc admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:14.314880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 827/1647] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:14 2022] PUT /identity/v3/projects/5c5275f0853a431f9d13a9764fec8b30/users/1d77393c5a2b47f8811d189bfc67f82f/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:14.325265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d28b-f8ba-4eb7-94c9-96a2769787ed None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.325622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d28b-f8ba-4eb7-94c9-96a2769787ed None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.325961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d28b-f8ba-4eb7-94c9-96a2769787ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:14.398021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b083d28b-f8ba-4eb7-94c9-96a2769787ed None None] MFA Rules not processed for user `1d77393c5a2b47f8811d189bfc67f82f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:14.400213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b083d28b-f8ba-4eb7-94c9-96a2769787ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:14.428151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:14.429029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.429238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.429489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] PATH_INFO: `/v3/users/d49986e88ae049bc8146010dd3a79af2` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:14.430717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d49986e88ae049bc8146010dd3a79af2)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:14.432763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:14.506937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 828/1648] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:14.509311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083d28b-f8ba-4eb7-94c9-96a2769787ed None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:14.526440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083d28b-f8ba-4eb7-94c9-96a2769787ed None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:14.534897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:14.536055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.536570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.537376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] PATH_INFO: `/v3/users/00c02d5ccb7d44dbab9d2846df39b025` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:14.540413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c02d5ccb7d44dbab9d2846df39b025)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:14.542062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: WARNING py.warnings [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:50:14.542062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: warnings.warn(msg) Aug 12 10:50:14.542062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: Aug 12 10:50:14.544120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:14.546488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd49986e88ae049bc8146010dd3a79af2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.561086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd49986e88ae049bc8146010dd3a79af2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.567449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd49986e88ae049bc8146010dd3a79af2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.593215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd49986e88ae049bc8146010dd3a79af2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.631108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd49986e88ae049bc8146010dd3a79af2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.660872 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 821/1649] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:14 2022] DELETE /identity/v3/users/d49986e88ae049bc8146010dd3a79af2 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:14.662286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:14.671021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:14.704443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c02d5ccb7d44dbab9d2846df39b025'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.709406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c02d5ccb7d44dbab9d2846df39b025'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.709406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c02d5ccb7d44dbab9d2846df39b025'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.784109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c02d5ccb7d44dbab9d2846df39b025'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.784552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:14.784552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447b0c-54fd-48a1-8a80-43ce9c34fd4d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:14.784552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:14.784552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.785109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.785109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:14.785109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:14.790696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-60d44ac9-103d-4a20-8d66-dbd26c34b5ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c02d5ccb7d44dbab9d2846df39b025'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:14.845832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:14.862793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 829/1650] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:14 2022] DELETE /identity/v3/users/00c02d5ccb7d44dbab9d2846df39b025 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:14.864982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff402d-bd4a-4c93-8dc8-15e14938e920 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.865248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff402d-bd4a-4c93-8dc8-15e14938e920 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.865536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff402d-bd4a-4c93-8dc8-15e14938e920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:14.892052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-eaff402d-bd4a-4c93-8dc8-15e14938e920 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:14.895058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-eaff402d-bd4a-4c93-8dc8-15e14938e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:14.973143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:14.978402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:14.986220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 830/1651] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:14.986220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:14.986220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:14.986220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:15.014788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 822/1652] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:15.020461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:15.022014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:15.034052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 831/1653] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:15.054647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:15.057285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:15.301469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5292396-66d9-449a-8a4e-237c27f6bd89 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:15.312371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:15.312371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:15.427809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c1b3c-f311-4826-a100-52e968a1e500 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:15.439031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:15.440527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:15.441296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:15.441851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:15.480001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:15.483050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:15.484569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:15.485078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:15.485817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:15.491210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:15.519143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 832/1654] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:15.546609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:15.552245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:15.608579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 823/1655] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:16.504216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:16.505834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:16.629092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92117e-26a3-482a-8886-9285b9a36ddc service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:16.639016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:16.639660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:16.639937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:16.640203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:16.653334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:16.741439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:16.743580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:16.766781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ebd1-84d3-4b19-9a61-39b797396744 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:16.779337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:16.779791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:16.780219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:16.781384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:16.781913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] PATH_INFO: `/v3/users/39e66fc5b67f4c17af43b6d1fe134dc4` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:16.784736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e66fc5b67f4c17af43b6d1fe134dc4)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:16.788596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:16.803127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:16.809254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:16.831801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 833/1656] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:16.898200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:16.900229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:16.925643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e66fc5b67f4c17af43b6d1fe134dc4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:16.928791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:16.929902 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a54426-d430-4f0e-b3e6-51cb7eee27a6 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:16.944066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:16.948227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:16.948906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:16.950078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] PATH_INFO: `/v3/projects/e778a4db38cd407092cb639b9738e35a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:16.950368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e66fc5b67f4c17af43b6d1fe134dc4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:16.952117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e66fc5b67f4c17af43b6d1fe134dc4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:16.955838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e778a4db38cd407092cb639b9738e35a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:16.959290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:16.976226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e66fc5b67f4c17af43b6d1fe134dc4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:17.014086 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84071052-f6c5-4229-918d-d2a99425fd2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e66fc5b67f4c17af43b6d1fe134dc4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:17.029995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 824/1657] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:16 2022] DELETE /identity/v3/users/39e66fc5b67f4c17af43b6d1fe134dc4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:50:17.036495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.038102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.047363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.070471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e778a4db38cd407092cb639b9738e35a', 'request_id': 'req-0aa507d4-120c-46f3-ad66-af1ae93f8451'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:17.085470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:17.087416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:17.089060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e778a4db38cd407092cb639b9738e35a', 'request_id': 'req-0aa507d4-120c-46f3-ad66-af1ae93f8451'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:17.103001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e778a4db38cd407092cb639b9738e35a', 'request_id': 'req-0aa507d4-120c-46f3-ad66-af1ae93f8451'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:17.148420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] The token cache is being invalidate because project e778a4db38cd407092cb639b9738e35a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:17.149245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:17.151740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0aa507d4-120c-46f3-ad66-af1ae93f8451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:17.159981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 834/1658] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:16 2022] DELETE /identity/v3/projects/e778a4db38cd407092cb639b9738e35a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:17.168093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af648c-de22-40f6-92dd-f1cf04e1fea6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.169182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af648c-de22-40f6-92dd-f1cf04e1fea6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.169544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af648c-de22-40f6-92dd-f1cf04e1fea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.232150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-99af648c-de22-40f6-92dd-f1cf04e1fea6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:17.235414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-99af648c-de22-40f6-92dd-f1cf04e1fea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:17.262986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 825/1659] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:17.382680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:17.384176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:17.418924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 835/1660] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:17.421320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc7a6-287f-475e-b636-ad6d85d1f03f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.421624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc7a6-287f-475e-b636-ad6d85d1f03f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.421896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc7a6-287f-475e-b636-ad6d85d1f03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.484286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3cbbc7a6-287f-475e-b636-ad6d85d1f03f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:17.486824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3cbbc7a6-287f-475e-b636-ad6d85d1f03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:17.512157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 836/1661] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:17.516891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbc7a6-287f-475e-b636-ad6d85d1f03f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:17.538425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbc7a6-287f-475e-b636-ad6d85d1f03f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:17.555664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:17.556698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.557528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.558718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.561411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:17.563322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:17.566463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:17.588454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d228f-f778-4e9f-90bb-aee29a8c54eb None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:17.615101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:17.615916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.619763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.620365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.621296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 837/1662] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:50:17.625201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:17.641439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6034dde-3ca0-42e7-bd5c-57928d9d6990 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:17.652074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:17.653632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:17.660104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:17.661007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.661445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.661977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.662950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:17.665223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:17.679498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 826/1663] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:50:17.720158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 838/1664] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:17 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:17.724382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:17.738619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176f9e3-dd12-4d39-b03f-a7131f3f2cf6 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:17.836424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:17.838326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.838817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.839221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.840361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:17.843070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:17.873917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:17.898689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 827/1665] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:17 2022] POST /identity/v3/users => generated 303 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:17.903471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:17.903953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:17.906331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:17.923559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af069638-372b-4171-a646-41567864cfc3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:17.932189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6770975-c89c-4ed6-9e67-81891f9ad65d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:17.940058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:17.941246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.941778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.942275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] PATH_INFO: `/v3/projects/18fbf543ce304e6c8d7c758760d6c08d` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.943123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:17.943744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:17.944011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:17.944309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:17.944614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] RBAC: Authorizing `identity:delete_project(project_id=18fbf543ce304e6c8d7c758760d6c08d)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:17.944968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:17.946583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:17.947726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:17.969685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 828/1666] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:17.970511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:17.998917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5398b-bee3-4625-8b27-310ec0e8abfe admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:18.000285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:18.008286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:18.009374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:18.009654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:18.010084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:18.011143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:18.019078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:18.045344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 829/1667] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:18.055937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:18.058448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18fbf543ce304e6c8d7c758760d6c08d', 'request_id': 'req-e197581b-f150-4570-9998-f18c8c81560e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.084034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18fbf543ce304e6c8d7c758760d6c08d', 'request_id': 'req-e197581b-f150-4570-9998-f18c8c81560e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.094220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8f0f6-7222-43b2-956e-70f37b8a376a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:18.101842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18fbf543ce304e6c8d7c758760d6c08d', 'request_id': 'req-e197581b-f150-4570-9998-f18c8c81560e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.135581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] The token cache is being invalidate because project 18fbf543ce304e6c8d7c758760d6c08d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:18.135778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.136594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.138218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 839/1668] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:17 2022] DELETE /identity/v3/projects/18fbf543ce304e6c8d7c758760d6c08d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:18.144440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:18.147727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:18.200943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bcd75-195a-4a46-acb1-99bebcc7e0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:18.201947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bcd75-195a-4a46-acb1-99bebcc7e0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:18.202409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bcd75-195a-4a46-acb1-99bebcc7e0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:18.203140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bcd75-195a-4a46-acb1-99bebcc7e0b0 admin admin] PATH_INFO: `/v3/projects/1406e302c1ed452e8addd1fe7c120f4a/users/fe3450a9a8ce415b991235ad2c553b1f/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:18.224065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bcd75-195a-4a46-acb1-99bebcc7e0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1406e302c1ed452e8addd1fe7c120f4a, user_id=fe3450a9a8ce415b991235ad2c553b1f, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:18.227988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bcd75-195a-4a46-acb1-99bebcc7e0b0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:18.273402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 830/1669] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:18 2022] PUT /identity/v3/projects/1406e302c1ed452e8addd1fe7c120f4a/users/fe3450a9a8ce415b991235ad2c553b1f/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:18.278895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:18.279300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:18.279787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:18.315058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197581b-f150-4570-9998-f18c8c81560e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:18.354596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:18.358300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:18.528656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:18.529298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:18.529298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:18.529298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] PATH_INFO: `/v3/users/7fa3b07d377f49ee8f01f425efbf6576` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:18.533301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa3b07d377f49ee8f01f425efbf6576)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:18.540674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:18.568863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 831/1670] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:18.614644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:18.616117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:18.716226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa3b07d377f49ee8f01f425efbf6576'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.718673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa3b07d377f49ee8f01f425efbf6576'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.725079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa3b07d377f49ee8f01f425efbf6576'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.767118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa3b07d377f49ee8f01f425efbf6576'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.769057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b2500-247f-4fd1-875b-f718cdeb7465 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:18.781818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40926a-55a2-444b-b6dc-a7a7e16286bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:18.782742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40926a-55a2-444b-b6dc-a7a7e16286bf service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:18.782853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40926a-55a2-444b-b6dc-a7a7e16286bf service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:18.783281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40926a-55a2-444b-b6dc-a7a7e16286bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:18.789778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1d40926a-55a2-444b-b6dc-a7a7e16286bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:18.796875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa3b07d377f49ee8f01f425efbf6576'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:18.800181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:18.840662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40926a-55a2-444b-b6dc-a7a7e16286bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:18.842162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40926a-55a2-444b-b6dc-a7a7e16286bf service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:18.853911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 840/1671] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:18 2022] DELETE /identity/v3/users/7fa3b07d377f49ee8f01f425efbf6576 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:18.864315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 832/1672] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:19.309842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:19.315251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:19.523097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d9c72-e1a4-40e6-ab0b-84597798822d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:19.543625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:19.544337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:19.544603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:19.544872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] PATH_INFO: `/v3/projects/09c9254199eb4abd9cd1f158e8485935` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:19.546410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c9254199eb4abd9cd1f158e8485935)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:19.548455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:19.716762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c9254199eb4abd9cd1f158e8485935', 'request_id': 'req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:19.729782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c9254199eb4abd9cd1f158e8485935', 'request_id': 'req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:19.737553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c9254199eb4abd9cd1f158e8485935', 'request_id': 'req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:19.766988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] The token cache is being invalidate because project 09c9254199eb4abd9cd1f158e8485935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:19.767296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:19.768318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-575f0aee-5fac-4ef9-ae93-0a7a8d8dd238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:19.773309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 841/1673] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:19 2022] DELETE /identity/v3/projects/09c9254199eb4abd9cd1f158e8485935 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:19.774996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665acb2b-16a1-4f76-8118-17d88817ab38 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:19.775292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665acb2b-16a1-4f76-8118-17d88817ab38 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:19.775567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665acb2b-16a1-4f76-8118-17d88817ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:19.812761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:19.860821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-665acb2b-16a1-4f76-8118-17d88817ab38 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:19.866432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-665acb2b-16a1-4f76-8118-17d88817ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:19.955928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 833/1674] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:19.963183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8339-f0f0-436f-be94-705ed5fab308 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:19.963460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8339-f0f0-436f-be94-705ed5fab308 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:19.963733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f8339-f0f0-436f-be94-705ed5fab308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:19.993462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-336f8339-f0f0-436f-be94-705ed5fab308 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:19.994717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-336f8339-f0f0-436f-be94-705ed5fab308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:20.019083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 842/1675] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:20.020816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665acb2b-16a1-4f76-8118-17d88817ab38 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.043067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665acb2b-16a1-4f76-8118-17d88817ab38 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.051984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.052650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.052931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.053271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.053947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:20.054241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:20.055885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:20.084374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 834/1676] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:20.089029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8339-f0f0-436f-be94-705ed5fab308 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.101744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f8339-f0f0-436f-be94-705ed5fab308 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.108740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.109363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.109635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.109898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.110532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:20.114384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:20.209221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 843/1677] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:20 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:20.213343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.239863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c186ede-693a-4491-9dd4-5b8e34917595 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.325055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.328397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:20.439625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:20.499220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.500815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.501688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.502270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.503261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:20.506138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72108b-6e3f-4f05-abcb-2751d2b143c3 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.507515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:20.515729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.516554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.516831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.517138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] PATH_INFO: `/v3/projects/0f26dfd61cb242d682b6e42552fb2786` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.533065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f26dfd61cb242d682b6e42552fb2786)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:20.535400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:20.550890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 835/1678] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:20 2022] POST /identity/v3/users => generated 284 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:20.551961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.567075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43a2fe-986e-46cf-9b01-a3b5a0e71b85 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.572527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.573275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.573620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.573997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.574902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:20.576896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:20.595112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 836/1679] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:20.596399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.611684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcfcf4-4687-47d7-bf4d-fe1d307363f1 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.657343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f26dfd61cb242d682b6e42552fb2786', 'request_id': 'req-d639d330-82f4-4d20-9ba9-3429f5538b85'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:20.685015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f26dfd61cb242d682b6e42552fb2786', 'request_id': 'req-d639d330-82f4-4d20-9ba9-3429f5538b85'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:20.702036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f26dfd61cb242d682b6e42552fb2786', 'request_id': 'req-d639d330-82f4-4d20-9ba9-3429f5538b85'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:20.728847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] The token cache is being invalidate because project 0f26dfd61cb242d682b6e42552fb2786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:20.729145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:20.732056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d639d330-82f4-4d20-9ba9-3429f5538b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:20.736106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 844/1680] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:20 2022] DELETE /identity/v3/projects/0f26dfd61cb242d682b6e42552fb2786 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:20.742816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d433c4-458d-46c7-af0a-2290ac7e80fe None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.743094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d433c4-458d-46c7-af0a-2290ac7e80fe None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.743360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d433c4-458d-46c7-af0a-2290ac7e80fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.805270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.807904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.808390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.809458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.810319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:20.816845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:20.824234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c6d433c4-458d-46c7-af0a-2290ac7e80fe None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:20.832064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c6d433c4-458d-46c7-af0a-2290ac7e80fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:20.855315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 845/1681] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:20.866867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 837/1682] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:20.867410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288ee8d-851a-45e2-8c02-57c6150e03c8 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.867410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288ee8d-851a-45e2-8c02-57c6150e03c8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.867410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288ee8d-851a-45e2-8c02-57c6150e03c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.876511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.884651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:20.904735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e288ee8d-851a-45e2-8c02-57c6150e03c8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:20.906001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e288ee8d-851a-45e2-8c02-57c6150e03c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:20.915682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:20.926506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc7ee2-8c6c-44d4-a917-db581929f32d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.940504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 846/1683] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:20.941746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288ee8d-851a-45e2-8c02-57c6150e03c8 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:20.949025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d374c3-feb4-4e04-8e6c-a488e45ca304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.950237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d374c3-feb4-4e04-8e6c-a488e45ca304 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.950728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d374c3-feb4-4e04-8e6c-a488e45ca304 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.951213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d374c3-feb4-4e04-8e6c-a488e45ca304 admin admin] PATH_INFO: `/v3/projects/6f4b66668a86430ba1862b765c3cfc98/users/7b202990c2f747abb29508eea13220c4/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:20.976229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288ee8d-851a-45e2-8c02-57c6150e03c8 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:20.990132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d374c3-feb4-4e04-8e6c-a488e45ca304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f4b66668a86430ba1862b765c3cfc98, user_id=7b202990c2f747abb29508eea13220c4, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:20.994007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d374c3-feb4-4e04-8e6c-a488e45ca304 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:20.994869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:20.995513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:20.996435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:20.996963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.000331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:21.002731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:21.004453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:21.026842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 838/1684] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:20 2022] PUT /identity/v3/projects/6f4b66668a86430ba1862b765c3cfc98/users/7b202990c2f747abb29508eea13220c4/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:21.037426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.038058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.038430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.044634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 847/1685] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:21.049188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:21.075062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b963f4-c92b-4445-8f32-d922ea770ffd admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:21.113925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] MFA Rules not processed for user `7b202990c2f747abb29508eea13220c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:21.115485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:21.265555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:21.266228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.266505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.266764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.268070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:21.270097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:21.358050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 848/1686] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:21 2022] POST /identity/v3/projects => generated 383 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:21.359366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:21.412621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 839/1687] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:21.426828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:21.433254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:21.485125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:21.577707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564452b7-ffef-4a0f-aee6-9dd64d0d48c5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:21.592011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:21.592931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.593219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.595645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.596780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:21.601216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:21.634948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00155f-541e-4342-9cf4-1b7603ef8327 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:21.647425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 849/1688] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:21 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:21.651366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:21.674209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780818d5-cb63-4212-81fd-b7736e64bb1b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:21.680339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:21.681092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.681462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.681825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.684588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:21.684588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:21.697543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 850/1689] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:21.713817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:21.731518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139caee-590e-4a61-b0d2-81900ad1e0bb admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:21.735768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92768dcb-fd05-4558-9f90-dfc97a4cd14a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:21.736425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92768dcb-fd05-4558-9f90-dfc97a4cd14a service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.736675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92768dcb-fd05-4558-9f90-dfc97a4cd14a service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.736940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92768dcb-fd05-4558-9f90-dfc97a4cd14a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.739337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:21.740320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.741054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.741521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.742415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:21.746450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:21.763827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 851/1690] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:21.769306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:21.797109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdfc33-3c4e-4030-9f73-36e8e7eec978 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:21.811109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:21.812306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.814099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.814793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] PATH_INFO: `/v3/projects/f1b514a5e2344c928fe3f28d86cc684c/users/058c0392ba2f429c90408f2e465b9777/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:21.839423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:21.851542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b514a5e2344c928fe3f28d86cc684c, user_id=058c0392ba2f429c90408f2e465b9777, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:21.860505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:21.866081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92768dcb-fd05-4558-9f90-dfc97a4cd14a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:21.868591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92768dcb-fd05-4558-9f90-dfc97a4cd14a service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:21.883807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 840/1691] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:21.896009 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 852/1692] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:21 2022] PUT /identity/v3/projects/f1b514a5e2344c928fe3f28d86cc684c/users/058c0392ba2f429c90408f2e465b9777/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:21.912880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e0f3-e4a5-404b-81a2-e402d3677d6e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:21.917895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e0f3-e4a5-404b-81a2-e402d3677d6e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:21.919211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e0f3-e4a5-404b-81a2-e402d3677d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:22.017300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-acc4e0f3-e4a5-404b-81a2-e402d3677d6e None None] MFA Rules not processed for user `058c0392ba2f429c90408f2e465b9777`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:22.018966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-acc4e0f3-e4a5-404b-81a2-e402d3677d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:22.136549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 841/1693] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:22.149638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:22.156472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:22.365800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52677b-047a-4963-a99c-22e3596c489d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:22.388763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577412c-bafc-41cb-be98-a062881fc1eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:22.391052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577412c-bafc-41cb-be98-a062881fc1eb service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:22.391052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577412c-bafc-41cb-be98-a062881fc1eb service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:22.391317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577412c-bafc-41cb-be98-a062881fc1eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:22.423549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577412c-bafc-41cb-be98-a062881fc1eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:22.427772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577412c-bafc-41cb-be98-a062881fc1eb service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:22.477907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 853/1694] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:23.321301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4e0f3-e4a5-404b-81a2-e402d3677d6e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:23.343786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4e0f3-e4a5-404b-81a2-e402d3677d6e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:23.354194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:23.355206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:23.355742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:23.356352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:23.361553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:23.480332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:23.482959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:23.513258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 842/1695] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:50:23.586102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256540-84dc-4108-8e14-3a7405239d7a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:23.586627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256540-84dc-4108-8e14-3a7405239d7a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:23.586780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256540-84dc-4108-8e14-3a7405239d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:23.642117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-77256540-84dc-4108-8e14-3a7405239d7a None None] MFA Rules not processed for user `fe3450a9a8ce415b991235ad2c553b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:23.644300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-77256540-84dc-4108-8e14-3a7405239d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:23.789662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 854/1696] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:23 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:23.791155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:23.794338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:23.948737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac984a-e0af-4f85-bc5a-1fb02327e7b4 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:23.957844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:23.959007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:23.959007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:23.959386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:23.959953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:23.963223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:24.002555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:24.026866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 843/1697] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:23 2022] POST /identity/v3/projects => generated 403 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:24.035197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77256540-84dc-4108-8e14-3a7405239d7a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:24.099669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77256540-84dc-4108-8e14-3a7405239d7a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:24.117879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:24.316429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:24.317272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:24.320453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:24.323263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:24.324159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:24.327350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:24.403846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:24.405633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 855/1698] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:24 2022] POST /identity/v3/users => generated 308 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:24.418761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e0132-e777-402b-9a5d-33d1c09ca51e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:24.436270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:24.436916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:24.437194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:24.437465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:24.438140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:24.440099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:24.473952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:24.477434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 844/1699] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:24.489949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3fb16-0ca6-4569-85b3-feaa1d37e0cf admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:24.506708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:24.508807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:24.508912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:24.509106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] PATH_INFO: `/v3/projects/210b5f2aff2d4d6184f12d3a2dbf1eba/users/97489669405643bcb6b496405449d220/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:24.583548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210b5f2aff2d4d6184f12d3a2dbf1eba, user_id=97489669405643bcb6b496405449d220, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:24.592767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:24.640887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 856/1700] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:24 2022] PUT /identity/v3/projects/210b5f2aff2d4d6184f12d3a2dbf1eba/users/97489669405643bcb6b496405449d220/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:24.642224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:24.666999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cba19a-ef38-43c2-a0cf-fc6e9ac27f5a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:24.830414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021113b-193f-42f8-8b06-88e0e69035b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:24.831032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021113b-193f-42f8-8b06-88e0e69035b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:24.831308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021113b-193f-42f8-8b06-88e0e69035b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:24.831574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021113b-193f-42f8-8b06-88e0e69035b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:24.832238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021113b-193f-42f8-8b06-88e0e69035b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:24.833885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021113b-193f-42f8-8b06-88e0e69035b9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:24.884166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 845/1701] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:24.888069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:24.914746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b24fa-74d6-4401-9f0f-883809a8c6d1 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:24.936986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b812a3-4101-4549-8b32-60b8353e7760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:24.952774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:24.953062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:24.953337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:24.957345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b812a3-4101-4549-8b32-60b8353e7760 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:24.957648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b812a3-4101-4549-8b32-60b8353e7760 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:24.959545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b812a3-4101-4549-8b32-60b8353e7760 admin admin] PATH_INFO: `/v3/projects/210b5f2aff2d4d6184f12d3a2dbf1eba/users/97489669405643bcb6b496405449d220/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:24.965287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b812a3-4101-4549-8b32-60b8353e7760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210b5f2aff2d4d6184f12d3a2dbf1eba, user_id=97489669405643bcb6b496405449d220, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:24.970206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b812a3-4101-4549-8b32-60b8353e7760 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:25.003365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 857/1702] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:24 2022] PUT /identity/v3/projects/210b5f2aff2d4d6184f12d3a2dbf1eba/users/97489669405643bcb6b496405449d220/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:25.023273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] MFA Rules not processed for user `58e209063f5c454d87d9f743bedfc734`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:25.036612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:25.202476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 846/1703] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:25.214644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870e6b0-6edb-4325-b00f-701b9c29bc39 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:25.214985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870e6b0-6edb-4325-b00f-701b9c29bc39 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:25.215177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870e6b0-6edb-4325-b00f-701b9c29bc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:25.270096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9870e6b0-6edb-4325-b00f-701b9c29bc39 None None] MFA Rules not processed for user `58e209063f5c454d87d9f743bedfc734`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:25.271335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9870e6b0-6edb-4325-b00f-701b9c29bc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:25.306728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 858/1704] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:25.315707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:25.318164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:25.573538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f86fe-7c77-4856-b81d-bd4433ecea72 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:25.598584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf213d03-5ba5-4e23-be72-8671632b7573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:25.600739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf213d03-5ba5-4e23-be72-8671632b7573 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:25.600739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf213d03-5ba5-4e23-be72-8671632b7573 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:25.600986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf213d03-5ba5-4e23-be72-8671632b7573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:25.615541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:25.628865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf213d03-5ba5-4e23-be72-8671632b7573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:25.631096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf213d03-5ba5-4e23-be72-8671632b7573 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:25.649181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 847/1705] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:26.946588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c974c02-5935-42f6-9d42-36609824db27 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:26.947138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c974c02-5935-42f6-9d42-36609824db27 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:26.947239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c974c02-5935-42f6-9d42-36609824db27 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:26.948702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 859/1706] 10.222.0.23 () {54 vars in 912 bytes} [Fri Aug 12 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:26.959207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cea84-ab87-4584-b16b-05205b917d09 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:26.959774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cea84-ab87-4584-b16b-05205b917d09 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:26.959845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cea84-ab87-4584-b16b-05205b917d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:27.011347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-5c1cea84-ab87-4584-b16b-05205b917d09 None None] MFA Rules not processed for user `a8f3a56366d348bfb0a5c445905dc46b`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:27.012679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5c1cea84-ab87-4584-b16b-05205b917d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:27.146857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 848/1707] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:50:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:27.203742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c974c02-5935-42f6-9d42-36609824db27 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:27.206646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2c974c02-5935-42f6-9d42-36609824db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:27.243582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c974c02-5935-42f6-9d42-36609824db27 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:27.263896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b39ae-44eb-4319-8746-576e369065d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:27.265660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b39ae-44eb-4319-8746-576e369065d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:27.265990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b39ae-44eb-4319-8746-576e369065d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:27.266264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b39ae-44eb-4319-8746-576e369065d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:27.287530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b39ae-44eb-4319-8746-576e369065d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:27.314942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b39ae-44eb-4319-8746-576e369065d0 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:27.380558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 860/1708] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:27.439742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97ff5-ab32-49d8-afd3-44d3dc66e42d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:27.440189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97ff5-ab32-49d8-afd3-44d3dc66e42d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:27.440537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97ff5-ab32-49d8-afd3-44d3dc66e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:27.510408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-5ad97ff5-ab32-49d8-afd3-44d3dc66e42d None None] MFA Rules not processed for user `97489669405643bcb6b496405449d220`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:27.511930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5ad97ff5-ab32-49d8-afd3-44d3dc66e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:27.680307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 849/1709] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:28.229934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459a13-5d20-461f-b224-fe23b513f067 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:28.230729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459a13-5d20-461f-b224-fe23b513f067 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:28.231166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459a13-5d20-461f-b224-fe23b513f067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:28.288173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-08459a13-5d20-461f-b224-fe23b513f067 None None] MFA Rules not processed for user `e0d5ff60d4b24fe0bbd771e8ed97db2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:28.290164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-08459a13-5d20-461f-b224-fe23b513f067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:28.384681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcef30-9a3c-4d38-8a6c-13c3a817a8bb None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:28.385252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcef30-9a3c-4d38-8a6c-13c3a817a8bb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:28.385252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcef30-9a3c-4d38-8a6c-13c3a817a8bb None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:28.386939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 850/1710] 10.222.0.23 () {54 vars in 961 bytes} [Fri Aug 12 10:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:28.394255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121db8a-4576-40b1-81b1-b8d5f6fbdbf7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:28.396393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121db8a-4576-40b1-81b1-b8d5f6fbdbf7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:28.398424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121db8a-4576-40b1-81b1-b8d5f6fbdbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:28.444121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3121db8a-4576-40b1-81b1-b8d5f6fbdbf7 None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:28.445775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3121db8a-4576-40b1-81b1-b8d5f6fbdbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:28.531232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 861/1711] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:28.546965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459a13-5d20-461f-b224-fe23b513f067 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:28.610302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459a13-5d20-461f-b224-fe23b513f067 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:28.638055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61860015-1a68-45a9-bd1b-fe2c2ea0ba73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:28.639136 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860015-1a68-45a9-bd1b-fe2c2ea0ba73 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:28.640045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860015-1a68-45a9-bd1b-fe2c2ea0ba73 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:28.640513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860015-1a68-45a9-bd1b-fe2c2ea0ba73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:28.659694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61860015-1a68-45a9-bd1b-fe2c2ea0ba73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:28.661644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61860015-1a68-45a9-bd1b-fe2c2ea0ba73 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:28.695697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 862/1712] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:28.698322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd208ed-2f14-41b4-816e-7f9b00d95c6a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:28.698887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd208ed-2f14-41b4-816e-7f9b00d95c6a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:28.700048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd208ed-2f14-41b4-816e-7f9b00d95c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:28.718451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 851/1713] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Aug 12 10:50:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:28.734009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121db8a-4576-40b1-81b1-b8d5f6fbdbf7 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:28.752013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121db8a-4576-40b1-81b1-b8d5f6fbdbf7 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:28.756982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5dd208ed-2f14-41b4-816e-7f9b00d95c6a None None] MFA Rules not processed for user `58e209063f5c454d87d9f743bedfc734`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:28.759249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5dd208ed-2f14-41b4-816e-7f9b00d95c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:28.767045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33961756-aa15-41c2-8c61-88e4c6436edb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:28.768240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961756-aa15-41c2-8c61-88e4c6436edb service placement] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:28.768632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961756-aa15-41c2-8c61-88e4c6436edb service placement] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:28.768967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33961756-aa15-41c2-8c61-88e4c6436edb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:28.775785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 863/1714] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:28.786560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-33961756-aa15-41c2-8c61-88e4c6436edb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:28.850072 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33961756-aa15-41c2-8c61-88e4c6436edb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:28.851692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33961756-aa15-41c2-8c61-88e4c6436edb service placement] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:28.858007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:28.894622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 852/1715] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:50:30.406210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d509f1-61ef-4f8b-9672-a88c78eaff1a None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:30.407188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d509f1-61ef-4f8b-9672-a88c78eaff1a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:30.407711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d509f1-61ef-4f8b-9672-a88c78eaff1a None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:30.409575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 864/1716] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:50:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:30.478248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:30.479109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:30.479611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:30.541495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:30.544390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:30.589837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 853/1717] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:50:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:30.616582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d509f1-61ef-4f8b-9672-a88c78eaff1a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:30.618998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a5d509f1-61ef-4f8b-9672-a88c78eaff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:30.684035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d509f1-61ef-4f8b-9672-a88c78eaff1a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:30.705721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c27ee-c7c8-4f92-a207-b287ef008698 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:30.709934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c27ee-c7c8-4f92-a207-b287ef008698 service placement] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:30.710459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c27ee-c7c8-4f92-a207-b287ef008698 service placement] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:30.711470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c27ee-c7c8-4f92-a207-b287ef008698 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:30.763748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:30.778382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c27ee-c7c8-4f92-a207-b287ef008698 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:30.780678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c27ee-c7c8-4f92-a207-b287ef008698 service placement] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:30.824666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:30.826081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:30.836179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 865/1718] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:31.017565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a5ad-8bd6-453d-9701-e3739e834f1f None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:31.018425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a5ad-8bd6-453d-9701-e3739e834f1f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:31.018875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a5ad-8bd6-453d-9701-e3739e834f1f None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:31.021241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 866/1719] 10.222.0.23 () {54 vars in 912 bytes} [Fri Aug 12 10:50:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 10:50:31.039356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfca99-f08b-4e93-acd7-c5f38a86748f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:31.039540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfca99-f08b-4e93-acd7-c5f38a86748f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:31.040954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfca99-f08b-4e93-acd7-c5f38a86748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:31.107400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aec35-feb2-41cf-a8cf-a77cbeedf3a7 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:31.112288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-85cfca99-f08b-4e93-acd7-c5f38a86748f None None] MFA Rules not processed for user `a8f3a56366d348bfb0a5c445905dc46b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:31.116037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-85cfca99-f08b-4e93-acd7-c5f38a86748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:31.123367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:31.127455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:31.127737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:31.128012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] PATH_INFO: `/v3/users/2bdd2c709e394c57a8d5cd221db552f9` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:31.156531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bdd2c709e394c57a8d5cd221db552f9)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:31.172595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:31.172802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 867/1720] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:50:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 12 10:50:31.300447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfca99-f08b-4e93-acd7-c5f38a86748f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:31.324462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfca99-f08b-4e93-acd7-c5f38a86748f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:31.358058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741d928-0664-4a35-be58-094a152afb4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:31.359710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741d928-0664-4a35-be58-094a152afb4b service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:31.361070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741d928-0664-4a35-be58-094a152afb4b service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:31.361904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741d928-0664-4a35-be58-094a152afb4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:31.397267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741d928-0664-4a35-be58-094a152afb4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:31.398843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741d928-0664-4a35-be58-094a152afb4b service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:31.403502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bdd2c709e394c57a8d5cd221db552f9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:31.423665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bdd2c709e394c57a8d5cd221db552f9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:31.425400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bdd2c709e394c57a8d5cd221db552f9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:31.439611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 868/1721] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:31.441619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bdd2c709e394c57a8d5cd221db552f9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:31.469534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bdd2c709e394c57a8d5cd221db552f9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:31.504597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 854/1722] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:30 2022] DELETE /identity/v3/users/2bdd2c709e394c57a8d5cd221db552f9 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:50:31.505231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c326c0d-ee28-4de5-b6e1-9b4e3d418769 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:31.505397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c326c0d-ee28-4de5-b6e1-9b4e3d418769 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:31.505657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c326c0d-ee28-4de5-b6e1-9b4e3d418769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:31.577469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3c326c0d-ee28-4de5-b6e1-9b4e3d418769 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:31.579742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3c326c0d-ee28-4de5-b6e1-9b4e3d418769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:31.770792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 869/1723] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:31.794793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:31.796401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:32.000966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faafe54-ec19-4da9-982b-4a0bb54a068d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:32.015601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed7bb7-c799-48fc-a22b-1e06e64e45b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:32.017093 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7bb7-c799-48fc-a22b-1e06e64e45b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:32.018437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7bb7-c799-48fc-a22b-1e06e64e45b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:32.018920 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed7bb7-c799-48fc-a22b-1e06e64e45b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:32.067342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed7bb7-c799-48fc-a22b-1e06e64e45b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:32.068929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed7bb7-c799-48fc-a22b-1e06e64e45b5 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:32.109667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 855/1724] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:33.035682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892428a5-6b4c-4c5c-8b87-72e217d15458 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:33.036211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892428a5-6b4c-4c5c-8b87-72e217d15458 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:33.036525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892428a5-6b4c-4c5c-8b87-72e217d15458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:33.202448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-892428a5-6b4c-4c5c-8b87-72e217d15458 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:33.204042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-892428a5-6b4c-4c5c-8b87-72e217d15458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.229393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:33.230229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:33.230640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:33.298950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:33.302650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] MFA Rules not processed for user `c0711a144b4c4432ab849367d3bd2b33`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:33.304378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.400072 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 870/1725] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:33.421595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0640f0-0e0a-4b5e-972a-03996a1d1b44 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:33.422804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0640f0-0e0a-4b5e-972a-03996a1d1b44 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:33.423482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0640f0-0e0a-4b5e-972a-03996a1d1b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:33.473616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 856/1726] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:33.491058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:33.492991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.509699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2c0640f0-0e0a-4b5e-972a-03996a1d1b44 None None] MFA Rules not processed for user `5c7c0993b20843e4804fa96cd9be34d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:33.512057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2c0640f0-0e0a-4b5e-972a-03996a1d1b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.595958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:33.685613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 871/1727] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:33.698986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:33.699273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:33.699550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:33.724022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ffa7-b6c7-4573-a4b3-8bddc80d291b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:33.738360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:33.739220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:33.739627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:33.740036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:33.740608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] MFA Rules not processed for user `5c7c0993b20843e4804fa96cd9be34d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:33.741868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.762715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 872/1728] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:33.770865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:33.773182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:33.785584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:33.787017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.789610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 857/1729] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:50:33.792185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:33.793968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.823374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6af272-a192-4444-bcb4-aab610f7ec94 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:33.842610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:33.843232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:33.843548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:33.843923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:33.869065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:33.871366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:33.920039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 873/1730] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:50:33.921114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:33.925228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:33.982839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b01a83-0d38-45c8-a168-767b18cf98f6 service nova] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:33.984174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d68ab-c004-4c05-a3ad-459eee8d54d0 service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:34.005437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5cfc8-7acf-4346-9385-5ef9168a74d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:34.008183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5cfc8-7acf-4346-9385-5ef9168a74d6 service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:34.008499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5cfc8-7acf-4346-9385-5ef9168a74d6 service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:34.009299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5cfc8-7acf-4346-9385-5ef9168a74d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:34.011223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:34.012012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:34.013191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:34.014382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] PATH_INFO: `/v3/users/058c0392ba2f429c90408f2e465b9777` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:34.017653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=058c0392ba2f429c90408f2e465b9777)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:34.033249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5cfc8-7acf-4346-9385-5ef9168a74d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:34.036180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5cfc8-7acf-4346-9385-5ef9168a74d6 service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:34.040891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:34.044427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:34.056958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 858/1731] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:50:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:50:34.183300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058c0392ba2f429c90408f2e465b9777'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:34.184983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058c0392ba2f429c90408f2e465b9777'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:34.187180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058c0392ba2f429c90408f2e465b9777'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:34.206100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058c0392ba2f429c90408f2e465b9777'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:34.235337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058c0392ba2f429c90408f2e465b9777'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:34.277809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 874/1732] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:33 2022] DELETE /identity/v3/users/058c0392ba2f429c90408f2e465b9777 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:34.283305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:34.283927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:34.284172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:34.337919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:34.348414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:34.547415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 859/1733] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:34.622387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:34.624414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:34.853582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ce38-649d-47b8-b085-fd819165bbd8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:34.863370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:34.866106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:34.868007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:34.868452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:34.889034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:34.890843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:34.901534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:34.910085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:34.922168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb32a2-4fc3-44d9-8bb9-db62475b0f5e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:34.935828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:34.940205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 875/1734] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:34.942403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:34.943406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:34.944291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] PATH_INFO: `/v3/users/fe3450a9a8ce415b991235ad2c553b1f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:34.947829 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3450a9a8ce415b991235ad2c553b1f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:34.954299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:35.034864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:35.036696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:35.060533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa4edc-a754-49dd-9d41-f407d835a3b4 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:35.068388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:35.069221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:35.069482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:35.069978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] PATH_INFO: `/v3/users/7b202990c2f747abb29508eea13220c4` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:35.071482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b202990c2f747abb29508eea13220c4)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:35.074426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:35.141796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3450a9a8ce415b991235ad2c553b1f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.144078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:35.167614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3450a9a8ce415b991235ad2c553b1f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.169363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3450a9a8ce415b991235ad2c553b1f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.188015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3450a9a8ce415b991235ad2c553b1f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.236335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3450a9a8ce415b991235ad2c553b1f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.246747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 860/1735] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:34 2022] DELETE /identity/v3/users/fe3450a9a8ce415b991235ad2c553b1f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:35.250879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:35.254403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:35.257504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b202990c2f747abb29508eea13220c4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.259217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b202990c2f747abb29508eea13220c4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.261616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b202990c2f747abb29508eea13220c4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.282834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b202990c2f747abb29508eea13220c4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.308874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-390ca0e9-869b-4c9f-b85b-4cdc17cb3402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b202990c2f747abb29508eea13220c4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.415062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 876/1736] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:35 2022] DELETE /identity/v3/users/7b202990c2f747abb29508eea13220c4 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:35.420897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:35.421192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:35.421495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:35.446395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc154c-1298-418f-9af7-a3fd000e0b3e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:35.471519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:35.517288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:35.530496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:35.540815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:35.541709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:35.542353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:35.542751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] PATH_INFO: `/v3/projects/dd9ae370235744339aedbbbe26570b22` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:35.544160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9ae370235744339aedbbbe26570b22)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:35.545990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: WARNING py.warnings [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 10:50:35.545990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: warnings.warn(msg) Aug 12 10:50:35.545990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: Aug 12 10:50:35.547654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:35.575056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 877/1737] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:35.711077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9ae370235744339aedbbbe26570b22', 'request_id': 'req-044832aa-a311-4b5f-8126-cce4d5212fff'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.728543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9ae370235744339aedbbbe26570b22', 'request_id': 'req-044832aa-a311-4b5f-8126-cce4d5212fff'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.732636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:35.734247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:35.783485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9ae370235744339aedbbbe26570b22', 'request_id': 'req-044832aa-a311-4b5f-8126-cce4d5212fff'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.841878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] The token cache is being invalidate because project dd9ae370235744339aedbbbe26570b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:35.842361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.843612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:35.845850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 861/1738] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:35 2022] DELETE /identity/v3/projects/dd9ae370235744339aedbbbe26570b22 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:35.851378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:35.853046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:35.979285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037397a-cf4c-4b60-8c25-7671143de7e8 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:36.119308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e720-c262-44d4-bc9b-7bccb52f911e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:36.120107 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e720-c262-44d4-bc9b-7bccb52f911e service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:36.120528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e720-c262-44d4-bc9b-7bccb52f911e service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:36.120897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e720-c262-44d4-bc9b-7bccb52f911e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:36.122750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044832aa-a311-4b5f-8126-cce4d5212fff admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:36.128796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cbc1e720-c262-44d4-bc9b-7bccb52f911e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:36.132132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:36.135360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:36.135640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:36.135921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] PATH_INFO: `/v3/users/fe5641a8c7bd4d29945dbab81bfdd2ca` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:36.137295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5641a8c7bd4d29945dbab81bfdd2ca)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:36.142233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:36.160699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e720-c262-44d4-bc9b-7bccb52f911e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:36.162379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e720-c262-44d4-bc9b-7bccb52f911e service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:36.193405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 878/1739] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:36.295804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5641a8c7bd4d29945dbab81bfdd2ca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:36.301190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:36.324544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5641a8c7bd4d29945dbab81bfdd2ca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:36.327957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5641a8c7bd4d29945dbab81bfdd2ca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:36.357598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5641a8c7bd4d29945dbab81bfdd2ca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:36.378128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5641a8c7bd4d29945dbab81bfdd2ca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:36.391215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 862/1740] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:35 2022] DELETE /identity/v3/users/fe5641a8c7bd4d29945dbab81bfdd2ca => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:37.306430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4e22-dee4-4347-a633-b0e62f97563f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:37.307573 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4e22-dee4-4347-a633-b0e62f97563f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:37.308232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4e22-dee4-4347-a633-b0e62f97563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:37.356064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-786f4e22-dee4-4347-a633-b0e62f97563f None None] MFA Rules not processed for user `a8f3a56366d348bfb0a5c445905dc46b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:37.358057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-786f4e22-dee4-4347-a633-b0e62f97563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:37.483998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 879/1741] 10.222.0.39 () {58 vars in 1015 bytes} [Fri Aug 12 10:50:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:37.751831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:37.753297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:37.776010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aeacd-7aae-4980-a262-8feb0d1ea932 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:37.793777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:37.795013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:37.795013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:37.799961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:37.825377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:37.825565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:37.825832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:37.840172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:37.841705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:37.863949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:37.890867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] MFA Rules not processed for user `e0d5ff60d4b24fe0bbd771e8ed97db2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:37.892137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:37.896184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 863/1742] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:38.026832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 880/1743] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:38.334238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:38.336208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:38.518708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775f7fe-6176-45d0-91ae-2168e27ce734 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:38.526100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:38.526896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:38.529034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:38.529335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] PATH_INFO: `/v3/projects/1406e302c1ed452e8addd1fe7c120f4a` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:38.531867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1406e302c1ed452e8addd1fe7c120f4a)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:38.536864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:38.669961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1406e302c1ed452e8addd1fe7c120f4a', 'request_id': 'req-f61bce19-de08-46bf-9cfb-6368401b6091'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:38.680978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:38.682189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:38.698354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1406e302c1ed452e8addd1fe7c120f4a', 'request_id': 'req-f61bce19-de08-46bf-9cfb-6368401b6091'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:38.715613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4862cb9-4406-431c-a4f9-eff84c450851 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:38.717703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1406e302c1ed452e8addd1fe7c120f4a', 'request_id': 'req-f61bce19-de08-46bf-9cfb-6368401b6091'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:38.752931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] The token cache is being invalidate because project 1406e302c1ed452e8addd1fe7c120f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:38.753585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:38.760060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:38.771998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 864/1744] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:38 2022] DELETE /identity/v3/projects/1406e302c1ed452e8addd1fe7c120f4a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:38.776013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:38.778404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:38.954903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:38.955809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:38.956073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:38.956392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] PATH_INFO: `/v3/projects/daef9c084bf04f749f314c3a22e39ec7` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:38.958849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=daef9c084bf04f749f314c3a22e39ec7)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:38.964205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:38.977864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:39.009271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bce19-de08-46bf-9cfb-6368401b6091 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:39.018791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:39.019443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:39.019717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:39.019989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] PATH_INFO: `/v3/users/97489669405643bcb6b496405449d220` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:39.021262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97489669405643bcb6b496405449d220)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:39.026666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:39.149243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97489669405643bcb6b496405449d220'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.172520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97489669405643bcb6b496405449d220'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.173847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97489669405643bcb6b496405449d220'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.177285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daef9c084bf04f749f314c3a22e39ec7', 'request_id': 'req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.201154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97489669405643bcb6b496405449d220'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.206955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daef9c084bf04f749f314c3a22e39ec7', 'request_id': 'req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.225389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97489669405643bcb6b496405449d220'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.227792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daef9c084bf04f749f314c3a22e39ec7', 'request_id': 'req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.243906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 865/1745] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:38 2022] DELETE /identity/v3/users/97489669405643bcb6b496405449d220 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:39.244957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:39.249122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:39.287859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] The token cache is being invalidate because project daef9c084bf04f749f314c3a22e39ec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:39.290574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.291969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.302960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 881/1746] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:38 2022] DELETE /identity/v3/projects/daef9c084bf04f749f314c3a22e39ec7 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:39.305060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:39.310592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:39.428691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:39.457426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aed02-db7e-412a-8bdd-38367956b502 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:39.479762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:39.481187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:39.482870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:39.488769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] PATH_INFO: `/v3/projects/6f4b66668a86430ba1862b765c3cfc98` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:39.494448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f4b66668a86430ba1862b765c3cfc98)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:39.499631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:39.507716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845fd7a-bb65-4d8f-bf4a-919d6fc15f3e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:39.535910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:39.536904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:39.538231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:39.539965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] PATH_INFO: `/v3/projects/f1b514a5e2344c928fe3f28d86cc684c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:39.542392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b514a5e2344c928fe3f28d86cc684c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:39.545896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:39.627462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f4b66668a86430ba1862b765c3cfc98', 'request_id': 'req-07ba08b9-40a3-4e54-9965-9313958ddda5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.640446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f4b66668a86430ba1862b765c3cfc98', 'request_id': 'req-07ba08b9-40a3-4e54-9965-9313958ddda5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.652678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f4b66668a86430ba1862b765c3cfc98', 'request_id': 'req-07ba08b9-40a3-4e54-9965-9313958ddda5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.660689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:39.678885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b514a5e2344c928fe3f28d86cc684c', 'request_id': 'req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.680717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] The token cache is being invalidate because project 6f4b66668a86430ba1862b765c3cfc98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:39.685957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.686826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.688938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 866/1747] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:39 2022] DELETE /identity/v3/projects/6f4b66668a86430ba1862b765c3cfc98 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:39.693151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:39.694625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:39.699871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b514a5e2344c928fe3f28d86cc684c', 'request_id': 'req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.727010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b514a5e2344c928fe3f28d86cc684c', 'request_id': 'req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.787432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] The token cache is being invalidate because project f1b514a5e2344c928fe3f28d86cc684c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:39.798976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.807433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fe4d6dcc-3f02-4295-a8b4-c78e897c29e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:39.808712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 882/1748] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:39 2022] DELETE /identity/v3/projects/f1b514a5e2344c928fe3f28d86cc684c => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:39.815385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44716c-5786-4c01-86db-734fb7780b70 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:39.817860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44716c-5786-4c01-86db-734fb7780b70 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:39.818295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44716c-5786-4c01-86db-734fb7780b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:39.882493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9c44716c-5786-4c01-86db-734fb7780b70 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:39.884822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9c44716c-5786-4c01-86db-734fb7780b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:39.906992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba08b9-40a3-4e54-9965-9313958ddda5 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:40.024217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 883/1749] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:40.026095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fbea4-17a8-4046-b875-bb5158116745 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:40.026685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fbea4-17a8-4046-b875-bb5158116745 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:40.027062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fbea4-17a8-4046-b875-bb5158116745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:40.071982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cc2fbea4-17a8-4046-b875-bb5158116745 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:40.076223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cc2fbea4-17a8-4046-b875-bb5158116745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:40.106752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 884/1750] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:40.113907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a255e-4e21-4c1d-a779-b69379de6785 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:40.114675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a255e-4e21-4c1d-a779-b69379de6785 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:40.115976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a255e-4e21-4c1d-a779-b69379de6785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:40.178888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c35a255e-4e21-4c1d-a779-b69379de6785 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:40.180079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c35a255e-4e21-4c1d-a779-b69379de6785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:40.198727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:40.199550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:40.199655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:40.203688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] PATH_INFO: `/v3/users/1d77393c5a2b47f8811d189bfc67f82f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:40.207281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d77393c5a2b47f8811d189bfc67f82f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:40.214101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 885/1751] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:40.214101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1562dc-7194-4dce-88c5-6cc62e013796 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:40.218636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1562dc-7194-4dce-88c5-6cc62e013796 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:40.220122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1562dc-7194-4dce-88c5-6cc62e013796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:40.226013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:40.296951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-4b1562dc-7194-4dce-88c5-6cc62e013796 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:40.298230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4b1562dc-7194-4dce-88c5-6cc62e013796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:40.335287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 886/1752] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:40.346891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1562dc-7194-4dce-88c5-6cc62e013796 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:40.377304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1562dc-7194-4dce-88c5-6cc62e013796 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:40.391557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:40.392375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:40.392653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:40.395533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:40.397224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:40.397224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:40.414358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d77393c5a2b47f8811d189bfc67f82f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:40.423043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:40.461351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d77393c5a2b47f8811d189bfc67f82f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:40.462277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d77393c5a2b47f8811d189bfc67f82f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:40.476052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 887/1753] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:40.478390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:40.481215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d77393c5a2b47f8811d189bfc67f82f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:40.529451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7cae0-84fa-47c1-ae05-784d255d9acb admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:40.549845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:40.550392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:40.550646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:40.554732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:40.555702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:40.556047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:40.563980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d77393c5a2b47f8811d189bfc67f82f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:40.567841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:40.597678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 867/1754] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:39 2022] DELETE /identity/v3/users/1d77393c5a2b47f8811d189bfc67f82f => generated 0 bytes in 902 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:40.598476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:40.601297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:40.624354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 888/1755] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:40.630016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:40.634651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:40.715853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06d7d-d5c1-48ce-9e79-8978df218f4a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:40.773392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:40.836114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:40.942956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:40.944578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:40.948890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:40.949337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:40.950137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:40.953710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:41.001123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc1af6-b571-4c66-aafb-c3c2d0837f0a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:41.041688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:41.046657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:41.046770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:41.049159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:41.053488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:41.061495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:41.093479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 889/1756] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:40 2022] POST /identity/v3/projects => generated 371 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:41.101108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:41.206663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 868/1757] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:40 2022] POST /identity/v3/projects => generated 369 bytes in 611 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:41.207652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:41.358234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2545a0-10ed-4173-806f-d07c960f7f76 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:41.568248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:41.571352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:41.571627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:41.571975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:41.572784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:41.574810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:41.599303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f607-d606-4c1e-b8d5-099ec26736ef admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:41.629441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:41.630097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:41.630416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:41.630734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:41.631376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:41.645709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:41.695578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 890/1758] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:41 2022] POST /identity/v3/users => generated 286 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:41.696997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:41.703646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:41.730939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 869/1759] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:41 2022] POST /identity/v3/users => generated 285 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:41.730939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:41.753656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa876155-82a7-47bf-877c-34e566c26536 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:41.774598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff88cf-c0d4-4edb-96bd-23e459ebf9b4 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:41.782147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:41.784326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:41.796190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:41.796948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] PATH_INFO: `/v3/projects/210b5f2aff2d4d6184f12d3a2dbf1eba` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:41.802119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=210b5f2aff2d4d6184f12d3a2dbf1eba)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:41.808979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:41.815331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:41.815877 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:41.816464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:41.817026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:41.818000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:41.820009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:41.852638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 870/1760] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:41.855596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:41.884106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94ecb0-62a0-44b1-9360-35c4271ad72c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:41.901298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:41.902345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:41.902788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:41.903193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:41.904031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:41.906389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:41.947710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 871/1761] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:41.951069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:41.982082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6466a27-7fd0-40cb-9d5c-e53d0be19680 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:42.019007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:42.023362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.023637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.023901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:42.024890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:42.026307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:42.049608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:42.056916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210b5f2aff2d4d6184f12d3a2dbf1eba', 'request_id': 'req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.061070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 872/1762] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:42.061070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:42.078580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210b5f2aff2d4d6184f12d3a2dbf1eba', 'request_id': 'req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.090527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8dc64-e4b5-48b8-ab3f-f88a6e0ab40c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:42.096150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210b5f2aff2d4d6184f12d3a2dbf1eba', 'request_id': 'req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.133024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] The token cache is being invalidate because project 210b5f2aff2d4d6184f12d3a2dbf1eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:42.133378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.135583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.140372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 891/1763] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:41 2022] DELETE /identity/v3/projects/210b5f2aff2d4d6184f12d3a2dbf1eba => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:42.141039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:42.148832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:42.262615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:42.265116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.265116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.265116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:42.265116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:42.266297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:42.277982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:42.301513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 873/1764] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:42.303796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:42.317192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:42.343956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cda4a-1938-45fe-bb6a-8d75306c7d11 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:42.362910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:42.364248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.369011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.369290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] PATH_INFO: `/v3/projects/5c5275f0853a431f9d13a9764fec8b30` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:42.370775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c5275f0853a431f9d13a9764fec8b30)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:42.372744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:42.427461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ca4d4-7ed8-4c7c-9d33-e44f0e12aea2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:42.440294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:42.441429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.442317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.442317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] PATH_INFO: `/v3/projects/3161b29af3ee42f3826710e9948b1168/users/b4cea83d101e435d8d97df207761d053/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:42.495857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c5275f0853a431f9d13a9764fec8b30', 'request_id': 'req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.505809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3161b29af3ee42f3826710e9948b1168, user_id=b4cea83d101e435d8d97df207761d053, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:42.512710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c5275f0853a431f9d13a9764fec8b30', 'request_id': 'req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.516977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:42.522975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c5275f0853a431f9d13a9764fec8b30', 'request_id': 'req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.575757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] The token cache is being invalidate because project 5c5275f0853a431f9d13a9764fec8b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:42.577918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.580206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-8cafef2e-d1b9-4b7e-92f3-e3d541eb38c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:42.585973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 874/1765] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:42 2022] DELETE /identity/v3/projects/5c5275f0853a431f9d13a9764fec8b30 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:42.588523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 892/1766] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:42 2022] PUT /identity/v3/projects/3161b29af3ee42f3826710e9948b1168/users/b4cea83d101e435d8d97df207761d053/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:42.588775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04ecd2-1752-499e-aec1-b35f2c7ee6cd None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.589055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04ecd2-1752-499e-aec1-b35f2c7ee6cd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.589328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04ecd2-1752-499e-aec1-b35f2c7ee6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:42.591404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:42.596418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:42.631065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-bd04ecd2-1752-499e-aec1-b35f2c7ee6cd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:42.643993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bd04ecd2-1752-499e-aec1-b35f2c7ee6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:42.845035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3da95d-db2d-4365-9ce3-13034f7faa3d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:42.855572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 875/1767] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:42.857389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103f2fc-bbd3-45b4-b666-683774052b8d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.860848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103f2fc-bbd3-45b4-b666-683774052b8d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.861337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103f2fc-bbd3-45b4-b666-683774052b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:42.870950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e30c7-7614-4a7d-8e19-204b22849977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:42.871570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e30c7-7614-4a7d-8e19-204b22849977 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.871845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e30c7-7614-4a7d-8e19-204b22849977 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.872114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e30c7-7614-4a7d-8e19-204b22849977 admin admin] PATH_INFO: `/v3/projects/388f532a711c46469cd5f2f7d5740878/users/0b629c0ecc0d4b7694e3a1cf44b8d6ab/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:42.932043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e30c7-7614-4a7d-8e19-204b22849977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388f532a711c46469cd5f2f7d5740878, user_id=0b629c0ecc0d4b7694e3a1cf44b8d6ab, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:42.940627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e30c7-7614-4a7d-8e19-204b22849977 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:42.964157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3103f2fc-bbd3-45b4-b666-683774052b8d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:42.966082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3103f2fc-bbd3-45b4-b666-683774052b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:42.983850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 893/1768] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:42 2022] PUT /identity/v3/projects/388f532a711c46469cd5f2f7d5740878/users/0b629c0ecc0d4b7694e3a1cf44b8d6ab/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:42.993657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0909e6-4e50-425c-8037-add82f678aa0 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:42.993936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0909e6-4e50-425c-8037-add82f678aa0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:42.994356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0909e6-4e50-425c-8037-add82f678aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.113389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8b0909e6-4e50-425c-8037-add82f678aa0 None None] MFA Rules not processed for user `b4cea83d101e435d8d97df207761d053`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:43.120289 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8b0909e6-4e50-425c-8037-add82f678aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:43.188122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 876/1769] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:43.206348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c70b05-0e3d-41a6-9fe8-4d3be5d47ba7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:43.206689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c70b05-0e3d-41a6-9fe8-4d3be5d47ba7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:43.206903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c70b05-0e3d-41a6-9fe8-4d3be5d47ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.234640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f9c70b05-0e3d-41a6-9fe8-4d3be5d47ba7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:43.234881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f9c70b05-0e3d-41a6-9fe8-4d3be5d47ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:43.245122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:43.248381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 877/1770] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:43.252331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:43.252331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:43.254533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.341700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] MFA Rules not processed for user `0b629c0ecc0d4b7694e3a1cf44b8d6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:43.343657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:43.355815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:43.437643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 894/1771] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:43.443215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c0985-2e54-4284-83cc-89378f832e53 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:43.444054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c0985-2e54-4284-83cc-89378f832e53 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:43.444054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c0985-2e54-4284-83cc-89378f832e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.546158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-573c0985-2e54-4284-83cc-89378f832e53 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:43.547417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-573c0985-2e54-4284-83cc-89378f832e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:43.583413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 895/1772] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:43.584409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c0985-2e54-4284-83cc-89378f832e53 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:43.593942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 878/1773] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:43.599095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:43.607825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:43.618996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c0985-2e54-4284-83cc-89378f832e53 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:43.633752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:43.634229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:43.634544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:43.634890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.636799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:43.637322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:43.638738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:43.698591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 896/1774] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:43.699782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:43.731154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94db25-f3c4-4c35-9965-5eeeaff64cc2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:43.759428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:43.760632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:43.764452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:43.764936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.765816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:43.766488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:43.773886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:43.825296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 897/1775] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:43.827933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:43.889944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287e03-1c52-4b17-9d4c-304ca262983c admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:43.906651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:43.907529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:43.908305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:43.908305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.912358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:43.917411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea868335-4fc7-424d-8e04-41ea0e86fd39 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:43.918603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:43.925059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8571f3d80b0749729012353fcbbd9e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:43.945127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:43.945487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:43.953040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:43.986347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 898/1776] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:43 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:43.987901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:43.993538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:43.995436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:44.047713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 879/1777] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:44.048957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:44.088074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0880d2-b166-40c3-ad0b-46ea6ed2efd0 service glance] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:44.149678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15008163-b6a3-448d-91fe-1cfa370673df admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:44.170401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:44.171188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:44.171571 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:44.171943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:44.172712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:44.175445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:44.249313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 899/1778] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:43 2022] POST /identity/v3/projects => generated 365 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:44.250582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:44.353805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:44.355068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:44.360963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:44.362342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:44.364203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:44.377266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:44.514397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:44.516161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 880/1779] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:44 2022] POST /identity/v3/projects => generated 387 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:44.518571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:44.530681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bab5c6-5158-4541-8ffa-a61d06156564 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:44.569246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:44.782106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b18e1-f081-4eff-855f-e5d2608f4bbf admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:44.796373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:44.797044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:44.797427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:44.797778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:44.798693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:44.802919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:44.803227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:44.804106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:44.806009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:44.810075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:44.811014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:44.813279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:44.876980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 881/1780] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:44 2022] POST /identity/v3/users => generated 283 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:44.878342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:44.901994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 900/1781] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:44 2022] POST /identity/v3/users => generated 291 bytes in 652 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:44.903423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:44.905875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af409f-f1e4-4153-b9e0-f6964f08ce8c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:44.929163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:44.930657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:44.936716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:44.937304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:44.938195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:44.942415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96421f-793e-40aa-b7d4-722ea43b8796 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:44.954604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:44.959956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:44.960758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:44.961130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:44.961402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:44.962129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:44.964150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.015520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 901/1782] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:45.016771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.038552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72342d-7bb8-4423-8ecd-8b279efff3ee admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:45.049234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 882/1783] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:44 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:45.053234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.060161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:45.061654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.062331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.062618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.063339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:45.067033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.092837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 902/1784] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:45.094558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.123343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52f1fc-3186-4a59-a303-ce484f7c4a71 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:45.140451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:45.141600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.142147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.142639 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.144159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:45.146054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.171369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9253a2b-efd8-4fe6-b72c-94b6401683ae admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:45.173533 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 883/1785] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:45.174697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.193401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:45.194232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.194511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.194873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.199263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c814400-3603-48b7-9242-6a8ab8c2d7fa admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:45.200832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:45.208099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.233116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:45.233988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.234370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.234757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] PATH_INFO: `/v3/projects/4746e0ce75d84901876d7b2662a5c581/users/4d635ae3490247c3bc7e71108e3edc5e/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.240608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 903/1786] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:45.247088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.258362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4746e0ce75d84901876d7b2662a5c581, user_id=4d635ae3490247c3bc7e71108e3edc5e, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:45.261358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.297436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 884/1787] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:45 2022] PUT /identity/v3/projects/4746e0ce75d84901876d7b2662a5c581/users/4d635ae3490247c3bc7e71108e3edc5e/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:45.304726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.308360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3907-81e7-4372-b563-facc26fd4b45 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:45.551321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:45.553790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.555373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.555834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] PATH_INFO: `/v3/projects/ac70105bdd314431a46b409a5cb19869/users/2c0df5ebebfb4ab3b28258cc54721297/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.578134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85c04f-8d4a-4016-afbd-10d117dc496a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:45.584304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac70105bdd314431a46b409a5cb19869, user_id=2c0df5ebebfb4ab3b28258cc54721297, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:45.587616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.615555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6f2aa-251d-4409-816b-71f336384a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:45.618668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6f2aa-251d-4409-816b-71f336384a8b admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.620200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 904/1788] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:45 2022] PUT /identity/v3/projects/ac70105bdd314431a46b409a5cb19869/users/2c0df5ebebfb4ab3b28258cc54721297/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:45.621319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.623233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6f2aa-251d-4409-816b-71f336384a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.624441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6f2aa-251d-4409-816b-71f336384a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.625974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6f2aa-251d-4409-816b-71f336384a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:45.633278 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6f2aa-251d-4409-816b-71f336384a8b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.674116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 885/1789] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:45.684522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1d548-f758-4923-990d-9726736cf6e5 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.687475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1d548-f758-4923-990d-9726736cf6e5 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.687779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1d548-f758-4923-990d-9726736cf6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.757732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:45.774135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-15e1d548-f758-4923-990d-9726736cf6e5 None None] MFA Rules not processed for user `2c0df5ebebfb4ab3b28258cc54721297`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:45.780773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-15e1d548-f758-4923-990d-9726736cf6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:45.832242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808e735-1a1c-49d4-903b-2109f0b853d6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:45.851832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:45.854303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:45.854718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:45.855861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:45.856738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:45.863505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:45.890338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:45.945611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 905/1790] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:45.945611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.971989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 886/1791] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:45.972981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1d548-f758-4923-990d-9726736cf6e5 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:45.984859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10d2f-53bf-443a-94ff-ee9df67139a6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:46.000047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1d548-f758-4923-990d-9726736cf6e5 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:46.002502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206ad59-23f4-474f-acb1-fe54d125e71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:46.004490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206ad59-23f4-474f-acb1-fe54d125e71f admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.005718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206ad59-23f4-474f-acb1-fe54d125e71f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.005718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206ad59-23f4-474f-acb1-fe54d125e71f admin admin] PATH_INFO: `/v3/projects/fee07c7bf00842299ba28ebdffe6dc8e/users/35051544e71a445bb37afdfe6aa0cc77/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.013009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:46.013732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.013962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.014624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] PATH_INFO: `/v3/projects/4746e0ce75d84901876d7b2662a5c581/users/4d635ae3490247c3bc7e71108e3edc5e/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.021665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4746e0ce75d84901876d7b2662a5c581, user_id=4d635ae3490247c3bc7e71108e3edc5e, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:46.028445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:46.047722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206ad59-23f4-474f-acb1-fe54d125e71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee07c7bf00842299ba28ebdffe6dc8e, user_id=35051544e71a445bb37afdfe6aa0cc77, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:46.051534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206ad59-23f4-474f-acb1-fe54d125e71f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:46.057272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 887/1792] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:45 2022] PUT /identity/v3/projects/4746e0ce75d84901876d7b2662a5c581/users/4d635ae3490247c3bc7e71108e3edc5e/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:46.058253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:46.106777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 906/1793] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:45 2022] PUT /identity/v3/projects/fee07c7bf00842299ba28ebdffe6dc8e/users/35051544e71a445bb37afdfe6aa0cc77/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:46.112174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848af962-14aa-4788-ad9c-2dc7600312da None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.113711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848af962-14aa-4788-ad9c-2dc7600312da None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.114341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848af962-14aa-4788-ad9c-2dc7600312da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.215374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-848af962-14aa-4788-ad9c-2dc7600312da None None] MFA Rules not processed for user `4d635ae3490247c3bc7e71108e3edc5e`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:46.217212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-848af962-14aa-4788-ad9c-2dc7600312da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:46.234104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaadd66-be50-4709-aad1-32f8d8e72e05 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:46.433891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700ef5-4fda-46b5-bca2-5037e6d4940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:46.440463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ef5-4fda-46b5-bca2-5037e6d4940b admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.440746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ef5-4fda-46b5-bca2-5037e6d4940b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.441019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ef5-4fda-46b5-bca2-5037e6d4940b admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.441681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ef5-4fda-46b5-bca2-5037e6d4940b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:46.443624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ef5-4fda-46b5-bca2-5037e6d4940b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:46.461214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 907/1794] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:46.462753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88172fc-328f-4057-b36e-5f16878b4c74 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.464518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88172fc-328f-4057-b36e-5f16878b4c74 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.464897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88172fc-328f-4057-b36e-5f16878b4c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.574186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 888/1795] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:46 2022] POST /identity/v3/projects => generated 365 bytes in 516 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:46.577454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6d65e-94cf-4d9c-96c1-01d474018c70 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.577732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6d65e-94cf-4d9c-96c1-01d474018c70 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.578003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6d65e-94cf-4d9c-96c1-01d474018c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.580137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e88172fc-328f-4057-b36e-5f16878b4c74 None None] MFA Rules not processed for user `35051544e71a445bb37afdfe6aa0cc77`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:46.586185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e88172fc-328f-4057-b36e-5f16878b4c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:46.628828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-27b6d65e-94cf-4d9c-96c1-01d474018c70 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:46.633211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-27b6d65e-94cf-4d9c-96c1-01d474018c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:46.769769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 908/1796] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:46.771454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad97bf-bcf5-4307-8181-4fe35de6fef9 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.771740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad97bf-bcf5-4307-8181-4fe35de6fef9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.772007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad97bf-bcf5-4307-8181-4fe35de6fef9 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.773845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 909/1797] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:46.774598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad97bf-bcf5-4307-8181-4fe35de6fef9 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:46.776417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ccad97bf-bcf5-4307-8181-4fe35de6fef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:46.786417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:46.805224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 889/1798] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:50:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:46.811165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6d65e-94cf-4d9c-96c1-01d474018c70 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:46.879053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:46.949732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad97bf-bcf5-4307-8181-4fe35de6fef9 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:46.960281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54203d-f192-4214-9fd7-4c92a8b7eac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:46.960990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54203d-f192-4214-9fd7-4c92a8b7eac7 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.961249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54203d-f192-4214-9fd7-4c92a8b7eac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.961590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54203d-f192-4214-9fd7-4c92a8b7eac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.968090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6d65e-94cf-4d9c-96c1-01d474018c70 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:46.981713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:46.982731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:46.983012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:46.983280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:46.983914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:46.988846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:47.042928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 890/1799] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:46 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:47.043781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:47.066017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c352-5411-4314-ac8b-d68fdc968274 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:47.078056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:47.078696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:47.078970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:47.079267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:47.081591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:47.085442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:47.155856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 891/1800] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:47 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:47.156909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:47.159712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:47.189003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54203d-f192-4214-9fd7-4c92a8b7eac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:47.190621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54203d-f192-4214-9fd7-4c92a8b7eac7 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:47.234174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 910/1801] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:47.241450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12944aa6-6425-4d68-be70-c091828928ff None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:47.241723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12944aa6-6425-4d68-be70-c091828928ff None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:47.241990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12944aa6-6425-4d68-be70-c091828928ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:47.260605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8311046-8828-489b-91c4-a17f01840592 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:47.269607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:47.269607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:47.269607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:47.269607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:47.312305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-12944aa6-6425-4d68-be70-c091828928ff None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:47.313887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-12944aa6-6425-4d68-be70-c091828928ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:47.386009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:47.387946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:47.401208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 892/1802] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:50:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:50:47.402126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:47.429794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:47.538998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 911/1803] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:50:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:47.540300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12944aa6-6425-4d68-be70-c091828928ff None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:47.608927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892772-ec04-4f92-b272-3bd1d329a3b4 service swift] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:47.619753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:47.621202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:47.623065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:47.623461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:47.624172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:47.625980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:47.663678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 893/1804] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:47.664408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:47.669348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:47.680537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12944aa6-6425-4d68-be70-c091828928ff None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:47.692418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:47.724647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:47.725174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:47.727841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:47.737281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:47.740050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:47.832917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 912/1805] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:47 2022] POST /identity/v3/users => generated 301 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:47.834539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:47.848461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:47.871674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf957b9-785b-4380-a1f6-8d13fab4c013 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:47.895985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:47.897267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:47.897802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:47.898304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] PATH_INFO: `/v3/projects/7b2f6c1db09845538f960a83de4f9756/users/649d37b300b7489fa89d369e35a7ab94/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:47.900529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79e3f1-f294-4182-8a6c-75f78539df8a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:47.906192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:47.907029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:47.907573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:47.908739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:47.956419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2f6c1db09845538f960a83de4f9756, user_id=649d37b300b7489fa89d369e35a7ab94, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:47.961480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:47.994629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:47.996698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:48.008758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 894/1807] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:48.010021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.010133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 913/1807] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:47 2022] PUT /identity/v3/projects/7b2f6c1db09845538f960a83de4f9756/users/649d37b300b7489fa89d369e35a7ab94/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:48.011424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.024737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b782952-3af4-43ea-a4cb-e80afeaa537a service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:48.100381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:48.101367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:48.101757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:48.102024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:48.102809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:48.106889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:48.108873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:48.122640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 895/1808] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:48.126657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.144393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df016c43-a7ef-4ac7-9b73-05c222fe4b89 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:48.146699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a392fe-67e2-4957-9341-b1c584b6a15f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:48.149833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:48.150457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:48.150727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:48.150986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:48.151630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:48.155986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:48.158410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:48.159092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:48.159364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:48.159634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] PATH_INFO: `/v3/projects/333fb0d887b546e1b4389c63c0caba1f/users/961f08636f9c4953b50950e27f1a6e67/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:48.183921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 914/1809] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:48.190632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.207257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333fb0d887b546e1b4389c63c0caba1f, user_id=961f08636f9c4953b50950e27f1a6e67, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:48.215633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:48.217456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a9401-ef0a-460d-8d09-d9467047df5a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:48.232683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac420a30-531a-4f0d-877e-4d541b6df8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:48.233769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac420a30-531a-4f0d-877e-4d541b6df8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:48.234072 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac420a30-531a-4f0d-877e-4d541b6df8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:48.234443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac420a30-531a-4f0d-877e-4d541b6df8fc admin admin] PATH_INFO: `/v3/projects/7b2f6c1db09845538f960a83de4f9756/users/649d37b300b7489fa89d369e35a7ab94/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:48.241077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac420a30-531a-4f0d-877e-4d541b6df8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2f6c1db09845538f960a83de4f9756, user_id=649d37b300b7489fa89d369e35a7ab94, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:48.244921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 896/1810] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:48 2022] PUT /identity/v3/projects/333fb0d887b546e1b4389c63c0caba1f/users/961f08636f9c4953b50950e27f1a6e67/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:50:48.250259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.250429 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac420a30-531a-4f0d-877e-4d541b6df8fc admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:48.294114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 915/1811] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:48 2022] PUT /identity/v3/projects/7b2f6c1db09845538f960a83de4f9756/users/649d37b300b7489fa89d369e35a7ab94/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:48.300519 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ef14a-ee7a-414a-a5ec-77f3a8174e58 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:48.301111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ef14a-ee7a-414a-a5ec-77f3a8174e58 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:48.304411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ef14a-ee7a-414a-a5ec-77f3a8174e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:48.387028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-df6ef14a-ee7a-414a-a5ec-77f3a8174e58 None None] MFA Rules not processed for user `649d37b300b7489fa89d369e35a7ab94`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:48.388823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-df6ef14a-ee7a-414a-a5ec-77f3a8174e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:48.452763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37ad21-789c-424e-b11f-9fde9cacdcf9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:48.655704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:48.656356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:48.656634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:48.657120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:48.658765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:48.666127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:48.689051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 897/1812] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:48.690638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.697348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:48.701618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 916/1813] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:48.704542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ef14a-ee7a-414a-a5ec-77f3a8174e58 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.747871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ef14a-ee7a-414a-a5ec-77f3a8174e58 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:48.764025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:48.764979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:48.765461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:48.768971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] PATH_INFO: `/v3/projects/333fb0d887b546e1b4389c63c0caba1f/users/961f08636f9c4953b50950e27f1a6e67/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:48.775007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333fb0d887b546e1b4389c63c0caba1f, user_id=961f08636f9c4953b50950e27f1a6e67, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:48.787772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:48.826169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 917/1814] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:48 2022] PUT /identity/v3/projects/333fb0d887b546e1b4389c63c0caba1f/users/961f08636f9c4953b50950e27f1a6e67/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:48.828744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:48.905234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df794216-0a7c-405a-8e52-1825283ff760 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:48.925818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:48.985923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:49.067911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b43c8-5729-492c-baba-76d845466704 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:49.083580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac749c-1abb-48dc-9bc9-f1dc616b5b2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:49.086171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:49.086171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:49.086429 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac749c-1abb-48dc-9bc9-f1dc616b5b2d service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:49.086739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:49.086906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:49.087629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac749c-1abb-48dc-9bc9-f1dc616b5b2d service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:49.087902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac749c-1abb-48dc-9bc9-f1dc616b5b2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:49.119911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac749c-1abb-48dc-9bc9-f1dc616b5b2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:49.128186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac749c-1abb-48dc-9bc9-f1dc616b5b2d service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:49.177986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 898/1815] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:49.187144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402ebdd-e56c-43b9-9d63-43d88233e28e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:49.188275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402ebdd-e56c-43b9-9d63-43d88233e28e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:49.188275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402ebdd-e56c-43b9-9d63-43d88233e28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:49.310742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1402ebdd-e56c-43b9-9d63-43d88233e28e None None] MFA Rules not processed for user `961f08636f9c4953b50950e27f1a6e67`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:49.313225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1402ebdd-e56c-43b9-9d63-43d88233e28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:49.327693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:49.329337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:49.357617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 918/1816] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:49.534077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:49.537411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 899/1817] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:49 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:49.544470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226e75f-2b93-43d7-9e58-dff38dc4f2e8 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:49.552053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402ebdd-e56c-43b9-9d63-43d88233e28e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:49.554052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bb615-176c-48eb-b14c-526402907abe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:49.554847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bb615-176c-48eb-b14c-526402907abe service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:49.555187 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bb615-176c-48eb-b14c-526402907abe service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:49.555522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bb615-176c-48eb-b14c-526402907abe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:49.609462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402ebdd-e56c-43b9-9d63-43d88233e28e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:49.614854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:49.615685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:49.616153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:49.616651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:49.642726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:49.644832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:49.669657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 900/1818] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:49 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:49.685576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bb615-176c-48eb-b14c-526402907abe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:49.689752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bb615-176c-48eb-b14c-526402907abe service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:49.736151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 919/1819] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:49.922151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:50.074465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145213e4-26e6-4d84-9a28-5b5b467cfeb3 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:50.087156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:50.088379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:50.089702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:50.090155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] PATH_INFO: `/v3/users/35051544e71a445bb37afdfe6aa0cc77` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:50.092119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35051544e71a445bb37afdfe6aa0cc77)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:50.095347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:50.130608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:50.223750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35051544e71a445bb37afdfe6aa0cc77'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:50.240818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35051544e71a445bb37afdfe6aa0cc77'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:50.244504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35051544e71a445bb37afdfe6aa0cc77'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:50.255910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35051544e71a445bb37afdfe6aa0cc77'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:50.288330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35051544e71a445bb37afdfe6aa0cc77'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:50.308850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 901/1820] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:49 2022] DELETE /identity/v3/users/35051544e71a445bb37afdfe6aa0cc77 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:50.310460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e02a8-5f79-48e4-ad16-a1ab48b3dbb6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:50.311523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e02a8-5f79-48e4-ad16-a1ab48b3dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:50.312042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e02a8-5f79-48e4-ad16-a1ab48b3dbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:50.358241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d97e02a8-5f79-48e4-ad16-a1ab48b3dbb6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:50.359477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d97e02a8-5f79-48e4-ad16-a1ab48b3dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:50.465308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 920/1821] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:50.483426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:50.485486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:50.570425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccb95d-b009-49d1-92ff-545f846fd570 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:50.574656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e5f5b-9791-47b2-ab2f-f8767937f292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:50.575145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e5f5b-9791-47b2-ab2f-f8767937f292 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:50.575592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e5f5b-9791-47b2-ab2f-f8767937f292 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:50.576002 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e5f5b-9791-47b2-ab2f-f8767937f292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:50.589188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e5f5b-9791-47b2-ab2f-f8767937f292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:50.590737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e5f5b-9791-47b2-ab2f-f8767937f292 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:50.603334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 902/1822] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:50.719701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1decd-a56f-4777-8580-808a6c41513a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:50.721751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1decd-a56f-4777-8580-808a6c41513a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:50.721751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1decd-a56f-4777-8580-808a6c41513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:50.751178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0de1decd-a56f-4777-8580-808a6c41513a None None] MFA Rules not processed for user `c0711a144b4c4432ab849367d3bd2b33`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:50.752770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0de1decd-a56f-4777-8580-808a6c41513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:50.815588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:50.910669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 921/1823] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:51.365155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ed1cc-ace9-4f2f-9447-7a10ba487c5b None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:51.365707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ed1cc-ace9-4f2f-9447-7a10ba487c5b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:51.365707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ed1cc-ace9-4f2f-9447-7a10ba487c5b None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:51.367870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 903/1824] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:51.383591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825093ac-bccf-4e6e-90b8-c6871440f0fb None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:51.384145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825093ac-bccf-4e6e-90b8-c6871440f0fb None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:51.384145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825093ac-bccf-4e6e-90b8-c6871440f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:51.431433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-825093ac-bccf-4e6e-90b8-c6871440f0fb None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:51.436828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-825093ac-bccf-4e6e-90b8-c6871440f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:51.497081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ed1cc-ace9-4f2f-9447-7a10ba487c5b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:51.499738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-717ed1cc-ace9-4f2f-9447-7a10ba487c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:51.546418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ed1cc-ace9-4f2f-9447-7a10ba487c5b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:51.554240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:51.554910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:51.555318 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:51.555653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] PATH_INFO: `/v3/projects/fee07c7bf00842299ba28ebdffe6dc8e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:51.557431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee07c7bf00842299ba28ebdffe6dc8e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:51.559837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:51.638885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 922/1825] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:50:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:51.640045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee07c7bf00842299ba28ebdffe6dc8e', 'request_id': 'req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.657599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee07c7bf00842299ba28ebdffe6dc8e', 'request_id': 'req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.665971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee07c7bf00842299ba28ebdffe6dc8e', 'request_id': 'req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.682295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] The token cache is being invalidate because project fee07c7bf00842299ba28ebdffe6dc8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:51.682598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.683384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.686918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 904/1826] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:51 2022] DELETE /identity/v3/projects/fee07c7bf00842299ba28ebdffe6dc8e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:51.690976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825093ac-bccf-4e6e-90b8-c6871440f0fb None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:51.702451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825093ac-bccf-4e6e-90b8-c6871440f0fb None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:51.744611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:51.746057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:51.769262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:51.779343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:51.779992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:51.780293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:51.780596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] PATH_INFO: `/v3/users/961f08636f9c4953b50950e27f1a6e67` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:51.781994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=961f08636f9c4953b50950e27f1a6e67)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:51.783955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:51.852609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961f08636f9c4953b50950e27f1a6e67'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.853643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961f08636f9c4953b50950e27f1a6e67'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.854533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961f08636f9c4953b50950e27f1a6e67'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.863389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961f08636f9c4953b50950e27f1a6e67'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.867296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:51.875459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961f08636f9c4953b50950e27f1a6e67'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:51.892646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb432e0-7e03-4c0c-9ef8-a8f28094e72b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:51.915819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785135cd-4358-4d26-b550-839e21ea10fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:51.916484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785135cd-4358-4d26-b550-839e21ea10fb service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:51.916754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785135cd-4358-4d26-b550-839e21ea10fb service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:51.917021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785135cd-4358-4d26-b550-839e21ea10fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:51.921150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-785135cd-4358-4d26-b550-839e21ea10fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:51.943430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785135cd-4358-4d26-b550-839e21ea10fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:51.945486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785135cd-4358-4d26-b550-839e21ea10fb service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:51.954832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 923/1827] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:51 2022] DELETE /identity/v3/users/961f08636f9c4953b50950e27f1a6e67 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:51.957794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 905/1828] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:50:52.853870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:52.855400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:53.056811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48755a-f474-4283-9ac3-df70a2912fef admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:53.098954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:53.100248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:53.100510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:53.100786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] PATH_INFO: `/v3/users/0b629c0ecc0d4b7694e3a1cf44b8d6ab` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:53.104442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b629c0ecc0d4b7694e3a1cf44b8d6ab)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:53.109145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:53.194769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b629c0ecc0d4b7694e3a1cf44b8d6ab'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.195735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b629c0ecc0d4b7694e3a1cf44b8d6ab'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.196802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b629c0ecc0d4b7694e3a1cf44b8d6ab'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.210620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b629c0ecc0d4b7694e3a1cf44b8d6ab'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.223190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b629c0ecc0d4b7694e3a1cf44b8d6ab'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.254348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 924/1829] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:52 2022] DELETE /identity/v3/users/0b629c0ecc0d4b7694e3a1cf44b8d6ab => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:53.255540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:53.256014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:53.256083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:53.284721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:53.286811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:53.396172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 906/1830] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:53.419499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:53.421440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:53.591907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96751a91-800d-4691-b41d-01a61624abb5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:53.598335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec8be6-b929-4aa0-8df9-d431153c07b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:53.599012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8be6-b929-4aa0-8df9-d431153c07b6 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:53.599371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8be6-b929-4aa0-8df9-d431153c07b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:53.599595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8be6-b929-4aa0-8df9-d431153c07b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:53.612851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec8be6-b929-4aa0-8df9-d431153c07b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:53.614295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec8be6-b929-4aa0-8df9-d431153c07b6 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:53.621335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:53.628352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 925/1831] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:53.689771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:53.694215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:53.728610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1577f43-045c-459c-b293-ed42b426ba03 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:53.756633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:53.757940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:53.758401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:53.759428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] PATH_INFO: `/v3/projects/333fb0d887b546e1b4389c63c0caba1f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:53.762140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=333fb0d887b546e1b4389c63c0caba1f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:53.766428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:53.877879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333fb0d887b546e1b4389c63c0caba1f', 'request_id': 'req-209db419-8b2f-4e33-8994-d7af528d7daf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.884119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333fb0d887b546e1b4389c63c0caba1f', 'request_id': 'req-209db419-8b2f-4e33-8994-d7af528d7daf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.893724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333fb0d887b546e1b4389c63c0caba1f', 'request_id': 'req-209db419-8b2f-4e33-8994-d7af528d7daf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.953556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] The token cache is being invalidate because project 333fb0d887b546e1b4389c63c0caba1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:53.953923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.956241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-209db419-8b2f-4e33-8994-d7af528d7daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:53.957883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 907/1832] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:53 2022] DELETE /identity/v3/projects/333fb0d887b546e1b4389c63c0caba1f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:53.962971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1886f-a5d1-4c00-8c75-a18f2072585a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:53.963470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1886f-a5d1-4c00-8c75-a18f2072585a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:53.963551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1886f-a5d1-4c00-8c75-a18f2072585a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.008490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3ed1886f-a5d1-4c00-8c75-a18f2072585a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:54.012029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3ed1886f-a5d1-4c00-8c75-a18f2072585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:54.132329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 926/1833] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:54.139891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.140531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.141004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.168042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:54.169335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:54.180300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 908/1834] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:54.183521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1886f-a5d1-4c00-8c75-a18f2072585a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.214927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1886f-a5d1-4c00-8c75-a18f2072585a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.215020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.221181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:54.229788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.230416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.230691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.230957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.231607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:54.231904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.234795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.253731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b631f-f167-4ec3-aa59-fc58df0fe089 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.262589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.263758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 927/1835] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:54.263961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.265454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.265818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] PATH_INFO: `/v3/projects/388f532a711c46469cd5f2f7d5740878` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.268322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.269064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=388f532a711c46469cd5f2f7d5740878)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.271122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.283806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9a5f0-d68d-48fe-ae2e-0f8cf322a8f9 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.286267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:54.290597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.291776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.292231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.292831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.293657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.296290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.348962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 928/1836] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:54 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:54.350358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.372535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388f532a711c46469cd5f2f7d5740878', 'request_id': 'req-b171f80d-f03c-469e-8dfc-84bd98389aee'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.385020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388f532a711c46469cd5f2f7d5740878', 'request_id': 'req-b171f80d-f03c-469e-8dfc-84bd98389aee'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.397937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388f532a711c46469cd5f2f7d5740878', 'request_id': 'req-b171f80d-f03c-469e-8dfc-84bd98389aee'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.444478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] The token cache is being invalidate because project 388f532a711c46469cd5f2f7d5740878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:54.444835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.446612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.451618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 909/1837] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:54 2022] DELETE /identity/v3/projects/388f532a711c46469cd5f2f7d5740878 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:54.453344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.455243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:54.529440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c775b22-e8ed-435b-977d-777f516a99c9 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.536366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.537616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.537893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.538186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.538798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.540862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.569322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171f80d-f03c-469e-8dfc-84bd98389aee admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.578277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.578944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.579455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.580041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] PATH_INFO: `/v3/users/4d635ae3490247c3bc7e71108e3edc5e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.581871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d635ae3490247c3bc7e71108e3edc5e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.583867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.590017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 929/1838] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:54 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:54.591126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.596589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:54.611392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36354908-921b-42e3-9a83-9e2a4defca73 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.617202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.618068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.618344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.618598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.619247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.621617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.639817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 930/1839] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:50:54.640978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.652234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508c6ff-4826-4b81-b817-7767708db23e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.657569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.658269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.659131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.659394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.660635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.662308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.664207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d635ae3490247c3bc7e71108e3edc5e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.674691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d635ae3490247c3bc7e71108e3edc5e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.675623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d635ae3490247c3bc7e71108e3edc5e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.680567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 931/1840] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:54.684307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.695870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13666866-05c8-495b-8cec-00723b49fba8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.700910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d635ae3490247c3bc7e71108e3edc5e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.701945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.702561 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.702832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.703093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] PATH_INFO: `/v3/projects/31b2ee22b04a44ac85eb8381e19b95d3/users/397e87d2eeac4be6a7882cea2afcd67f/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.715620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a177c09f-fb56-4329-b47e-1e998cc33110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d635ae3490247c3bc7e71108e3edc5e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:54.722331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b2ee22b04a44ac85eb8381e19b95d3, user_id=397e87d2eeac4be6a7882cea2afcd67f, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.723291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 910/1841] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:54 2022] DELETE /identity/v3/users/4d635ae3490247c3bc7e71108e3edc5e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:54.725357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.741140 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 932/1842] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:54 2022] PUT /identity/v3/projects/31b2ee22b04a44ac85eb8381e19b95d3/users/397e87d2eeac4be6a7882cea2afcd67f/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:54.752757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.753043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.753460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.783162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] MFA Rules not processed for user `397e87d2eeac4be6a7882cea2afcd67f`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:54.784398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:54.794038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:54.857871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 911/1843] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:54.869077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.870838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:54.883398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:54.936684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97d2f9-3005-4555-ac9a-222ba04975ca admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:54.943145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:54.943750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:54.944704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:54.952592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:54.962373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:54.963839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:54.973208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 933/1844] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:54.992210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:54.993621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.016170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.018694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.070386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f5dfa-edf7-46cc-8bc3-60a912d99376 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.076226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.077070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.077583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.077939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] PATH_INFO: `/v3/users/397e87d2eeac4be6a7882cea2afcd67f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.079138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=397e87d2eeac4be6a7882cea2afcd67f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.081838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.113478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59dc0d-b40f-486c-852d-cc93dc6a9c37 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.120839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.121553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.121818 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.122214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] PATH_INFO: `/v3/projects/4746e0ce75d84901876d7b2662a5c581` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.124035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4746e0ce75d84901876d7b2662a5c581)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.126043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.184496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '397e87d2eeac4be6a7882cea2afcd67f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.195782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '397e87d2eeac4be6a7882cea2afcd67f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.197849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '397e87d2eeac4be6a7882cea2afcd67f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.206704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4746e0ce75d84901876d7b2662a5c581', 'request_id': 'req-dbb8aeba-6efd-466d-8973-9d84b499aa46'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.214474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4746e0ce75d84901876d7b2662a5c581', 'request_id': 'req-dbb8aeba-6efd-466d-8973-9d84b499aa46'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.215937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '397e87d2eeac4be6a7882cea2afcd67f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.222682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4746e0ce75d84901876d7b2662a5c581', 'request_id': 'req-dbb8aeba-6efd-466d-8973-9d84b499aa46'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.232939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-62dcfd24-8f31-46db-bd9b-e8743dfcbec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '397e87d2eeac4be6a7882cea2afcd67f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.239214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] The token cache is being invalidate because project 4746e0ce75d84901876d7b2662a5c581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:55.239580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.239700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 912/1845] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:50:54 2022] DELETE /identity/v3/users/397e87d2eeac4be6a7882cea2afcd67f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:55.240443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dbb8aeba-6efd-466d-8973-9d84b499aa46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:55.241953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 934/1846] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:55 2022] DELETE /identity/v3/projects/4746e0ce75d84901876d7b2662a5c581 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:55.246643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ced6-6079-4a16-9c8e-c777150c2e97 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.246931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ced6-6079-4a16-9c8e-c777150c2e97 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.247205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ced6-6079-4a16-9c8e-c777150c2e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.247470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.247757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.248036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.268317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9f85ced6-6079-4a16-9c8e-c777150c2e97 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:55.269567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9f85ced6-6079-4a16-9c8e-c777150c2e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.278342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:55.279524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.304695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:55.346275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:55.356596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 913/1847] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:55.367470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.369034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.380312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 935/1848] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:55.381927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c4dd-27d4-4bd5-b66e-646bfeb4fbb3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.382209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c4dd-27d4-4bd5-b66e-646bfeb4fbb3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.382477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c4dd-27d4-4bd5-b66e-646bfeb4fbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.403288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2d74c4dd-27d4-4bd5-b66e-646bfeb4fbb3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:55.404518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2d74c4dd-27d4-4bd5-b66e-646bfeb4fbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.413130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 936/1849] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:55.418585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74c4dd-27d4-4bd5-b66e-646bfeb4fbb3 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.433184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74c4dd-27d4-4bd5-b66e-646bfeb4fbb3 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.438721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.439706 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.440220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.440593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.441311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:55.441673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.443360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.460034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 937/1850] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:50:55.461159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.462594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724b47d-dc29-4fb6-ab47-f300b9f2ec52 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.466296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.466909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.467340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.468034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.473204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb063e-af73-4789-be91-387fc2a6b9ff admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.478296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.478459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.478934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.479212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.479483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.480069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.480204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.482022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.492932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 914/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:55.521814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 938/1852] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:55 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:55.523067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.533688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3186d0c-3c31-40e6-a037-7e4ee9fd0b9d service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.599608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.600577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.600855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.601121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.601762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.603714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.630034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 915/1853] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:55 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:55.631265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.647321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46548ef-e2b6-4220-8b8d-052595b41481 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.653819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.654733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.655004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.655269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.655934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.657758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.669697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 939/1854] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:50:55.674282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.685142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4d247-33d8-499e-9829-26f4f301761e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.690892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ed16f-2f40-4476-983b-5b8b981f82ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.691664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ed16f-2f40-4476-983b-5b8b981f82ca admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.691946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ed16f-2f40-4476-983b-5b8b981f82ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.692524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ed16f-2f40-4476-983b-5b8b981f82ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.692886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ed16f-2f40-4476-983b-5b8b981f82ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.694598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ed16f-2f40-4476-983b-5b8b981f82ca admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.706572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 916/1855] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:55.713735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.732592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b3b4-4458-49a0-bd04-35fb9a0e1232 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.737537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.738190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.738461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.738727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] PATH_INFO: `/v3/projects/5547c976d8a34145b474ff65bdd2e083/users/a446b137625b4f90b9c4bfda2e27a19e/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.756387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5547c976d8a34145b474ff65bdd2e083, user_id=a446b137625b4f90b9c4bfda2e27a19e, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.759114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.775979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 940/1856] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:55 2022] PUT /identity/v3/projects/5547c976d8a34145b474ff65bdd2e083/users/a446b137625b4f90b9c4bfda2e27a19e/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:55.786485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d6f7-cc88-48aa-8839-ccb8ef0e00c2 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.786798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d6f7-cc88-48aa-8839-ccb8ef0e00c2 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.787071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d6f7-cc88-48aa-8839-ccb8ef0e00c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.821368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3db8d6f7-cc88-48aa-8839-ccb8ef0e00c2 None None] MFA Rules not processed for user `a446b137625b4f90b9c4bfda2e27a19e`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:55.822598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3db8d6f7-cc88-48aa-8839-ccb8ef0e00c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.875055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:55.895973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.897709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:55.904274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 917/1857] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:55.908882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d6f7-cc88-48aa-8839-ccb8ef0e00c2 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:55.987938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec084b-2ee0-4857-9e9e-4d3d1db84d93 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.991435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d6f7-cc88-48aa-8839-ccb8ef0e00c2 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:55.992528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.993157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.993423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.993686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] PATH_INFO: `/v3/projects/31b2ee22b04a44ac85eb8381e19b95d3` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.994820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b2ee22b04a44ac85eb8381e19b95d3)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.995952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:55.996582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:55.996689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:55.996854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:55.997120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:55.997743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:55.999623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.031348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 918/1858] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:55 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:56.032466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.044155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b2ee22b04a44ac85eb8381e19b95d3', 'request_id': 'req-9157072b-3006-44d6-8dbc-5f435049e22c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:56.051099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b2ee22b04a44ac85eb8381e19b95d3', 'request_id': 'req-9157072b-3006-44d6-8dbc-5f435049e22c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:56.057073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b2ee22b04a44ac85eb8381e19b95d3', 'request_id': 'req-9157072b-3006-44d6-8dbc-5f435049e22c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:56.061043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:56.074997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] The token cache is being invalidate because project 31b2ee22b04a44ac85eb8381e19b95d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:50:56.075320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:56.076811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9157072b-3006-44d6-8dbc-5f435049e22c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:50:56.078563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 941/1859] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:50:55 2022] DELETE /identity/v3/projects/31b2ee22b04a44ac85eb8381e19b95d3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:56.083575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7edd-91ae-4c19-ac0f-cea8e5aa4dfa None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.083857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7edd-91ae-4c19-ac0f-cea8e5aa4dfa None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.084144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7edd-91ae-4c19-ac0f-cea8e5aa4dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.098061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e82df-46b6-4a26-afba-6b2586c49b4a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.103847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c71c7edd-91ae-4c19-ac0f-cea8e5aa4dfa None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:56.105067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c71c7edd-91ae-4c19-ac0f-cea8e5aa4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.160871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.161485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.161755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.162027 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.162650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.164643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.166030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 942/1860] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:56.173569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f9695-aa25-4836-b6ba-36e01a9f03a7 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.173893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f9695-aa25-4836-b6ba-36e01a9f03a7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.174110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f9695-aa25-4836-b6ba-36e01a9f03a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.194307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 919/1861] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:56 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:50:56.196141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-1f1f9695-aa25-4836-b6ba-36e01a9f03a7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:56.197385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1f1f9695-aa25-4836-b6ba-36e01a9f03a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.198408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.200144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.205531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 943/1862] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:56.207543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:56.209744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f9695-aa25-4836-b6ba-36e01a9f03a7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.214152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ca5b7-a108-422a-a721-820dbde2ff7a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.221595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.222138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f9695-aa25-4836-b6ba-36e01a9f03a7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.222264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.222400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.222663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.223320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.225034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.228614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.229261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.229537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.229949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.230607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:50:56.230902 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.233041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.240441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 920/1863] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:50:56.243033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.248425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 944/1864] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:50:56.251392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.261104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650adc-68be-4fb7-8fe2-f1ea73a48f0d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.261322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515c2d3-5de7-4546-88d2-183854bc44cd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.265733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.266366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.266747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.266849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.267123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.267518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.267747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.267837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.268393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] PATH_INFO: `/v3/projects/aa57958d455841ae99e48a4141c17aad/users/cc9905f9fcd74e11926165a307b6386f/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.269994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.285440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa57958d455841ae99e48a4141c17aad, user_id=cc9905f9fcd74e11926165a307b6386f, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.288251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.299070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 945/1865] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:50:56 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:56.302809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.304474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 921/1866] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:56 2022] PUT /identity/v3/projects/aa57958d455841ae99e48a4141c17aad/users/cc9905f9fcd74e11926165a307b6386f/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:56.308697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.385154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3634513-eec8-41ab-a3f0-95c6f8ae3e18 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.398142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e599-2398-482d-a41b-739b65226cef admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.403379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.404141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.404445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.404709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.405356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.406945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.430304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 922/1867] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:56.434461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.445238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4936a02-020a-479f-9cc4-5f2e369b049c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.452144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf09a64-625c-4345-acd2-c012b1c77e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.453156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf09a64-625c-4345-acd2-c012b1c77e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.453607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf09a64-625c-4345-acd2-c012b1c77e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.453883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf09a64-625c-4345-acd2-c012b1c77e3a admin admin] PATH_INFO: `/v3/projects/aa57958d455841ae99e48a4141c17aad/users/cc9905f9fcd74e11926165a307b6386f/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.457545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf09a64-625c-4345-acd2-c012b1c77e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa57958d455841ae99e48a4141c17aad, user_id=cc9905f9fcd74e11926165a307b6386f, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.460251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf09a64-625c-4345-acd2-c012b1c77e3a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.465271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.466545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.466545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.466830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.467419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.469728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.472524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 923/1868] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:56 2022] PUT /identity/v3/projects/aa57958d455841ae99e48a4141c17aad/users/cc9905f9fcd74e11926165a307b6386f/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:50:56.479757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:56.484139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d76f00-2bf8-49cb-8d4b-f24e3587ce61 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.484454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d76f00-2bf8-49cb-8d4b-f24e3587ce61 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.484727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d76f00-2bf8-49cb-8d4b-f24e3587ce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.494500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 946/1869] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:50:56 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:50:56.498661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.510451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30d1c8-4724-4802-8ff3-e3710993f134 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.517305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-e3d76f00-2bf8-49cb-8d4b-f24e3587ce61 None None] MFA Rules not processed for user `cc9905f9fcd74e11926165a307b6386f`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:56.518851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e3d76f00-2bf8-49cb-8d4b-f24e3587ce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.589652 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.590265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.590536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.590805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.592071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.593113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.599490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 924/1870] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:56.604960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 947/1871] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:50:56.608905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d76f00-2bf8-49cb-8d4b-f24e3587ce61 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.610670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.613175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.628675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d76f00-2bf8-49cb-8d4b-f24e3587ce61 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.633487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.634171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.634503 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.634847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.635538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.637245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.651896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 925/1872] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:50:56.653528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.665253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bb548-5060-410b-af1f-d88eb3e1ed63 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.670838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3672c-bc32-48ae-a826-3a1874c2ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.671486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3672c-bc32-48ae-a826-3a1874c2ff50 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.671758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3672c-bc32-48ae-a826-3a1874c2ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.672061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3672c-bc32-48ae-a826-3a1874c2ff50 admin admin] PATH_INFO: `/v3/projects/7795fcb0687b47aab64c34d161d24856/users/d17fb4b6648e436ab20481059f81600a/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.681815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1a753-0b56-4b33-9521-08a6cfc9ddc5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.685574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.686253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.691695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.691695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.697119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3672c-bc32-48ae-a826-3a1874c2ff50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7795fcb0687b47aab64c34d161d24856, user_id=d17fb4b6648e436ab20481059f81600a, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.699828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3672c-bc32-48ae-a826-3a1874c2ff50 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.702476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.703990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.708855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:56.713494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 926/1873] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:50:56 2022] PUT /identity/v3/projects/7795fcb0687b47aab64c34d161d24856/users/d17fb4b6648e436ab20481059f81600a/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:50:56.717388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 948/1874] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:56.720523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be0f3-dae9-4979-886e-f54daf96338a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.720808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be0f3-dae9-4979-886e-f54daf96338a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.721079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be0f3-dae9-4979-886e-f54daf96338a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.742158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a29be0f3-dae9-4979-886e-f54daf96338a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:56.743405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a29be0f3-dae9-4979-886e-f54daf96338a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.812105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 927/1875] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:50:56.822253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.824098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.830921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be0f3-dae9-4979-886e-f54daf96338a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:56.896742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be0f3-dae9-4979-886e-f54daf96338a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.897495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e192-b353-4a57-b4a3-ad9b8aee6827 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:56.900501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80da116-b8e2-4706-a232-54cb1a6ce1b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.901110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80da116-b8e2-4706-a232-54cb1a6ce1b6 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.901392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80da116-b8e2-4706-a232-54cb1a6ce1b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.901664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80da116-b8e2-4706-a232-54cb1a6ce1b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.903528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a80da116-b8e2-4706-a232-54cb1a6ce1b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:56.947279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:56.957545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c0416-ef50-48e6-810e-6b3ec29ad300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:56.958469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c0416-ef50-48e6-810e-6b3ec29ad300 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:56.959851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c0416-ef50-48e6-810e-6b3ec29ad300 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:56.959851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c0416-ef50-48e6-810e-6b3ec29ad300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:56.977911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c0416-ef50-48e6-810e-6b3ec29ad300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.979407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c0416-ef50-48e6-810e-6b3ec29ad300 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.982773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80da116-b8e2-4706-a232-54cb1a6ce1b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:56.985800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80da116-b8e2-4706-a232-54cb1a6ce1b6 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:56.993028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 949/1876] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:57.000850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 928/1877] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:50:57.194506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1b29a-bbbc-4d00-b012-95f5266c0e48 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:57.194817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1b29a-bbbc-4d00-b012-95f5266c0e48 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:57.196207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1b29a-bbbc-4d00-b012-95f5266c0e48 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:57.199572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 950/1878] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:50:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:50:57.215760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a26f7-21a8-4aff-b21b-e8385fd79c44 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:57.216080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a26f7-21a8-4aff-b21b-e8385fd79c44 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:57.216451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a26f7-21a8-4aff-b21b-e8385fd79c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:57.257994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-c12a26f7-21a8-4aff-b21b-e8385fd79c44 None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:57.263913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c12a26f7-21a8-4aff-b21b-e8385fd79c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:57.447795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:50:57.515264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 929/1879] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:50:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:57.531257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1b29a-bbbc-4d00-b012-95f5266c0e48 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:57.534697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-45b1b29a-bbbc-4d00-b012-95f5266c0e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:57.578092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1b29a-bbbc-4d00-b012-95f5266c0e48 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:57.593967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:57.595837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:57.595837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:57.595837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:57.607711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:57.609336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:57.628038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 951/1880] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:50:58.251203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb773-cdc1-4994-ad30-29d1cf35ffac None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:58.252107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb773-cdc1-4994-ad30-29d1cf35ffac None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:58.252428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cb773-cdc1-4994-ad30-29d1cf35ffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:58.283531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-363cb773-cdc1-4994-ad30-29d1cf35ffac None None] MFA Rules not processed for user `d17fb4b6648e436ab20481059f81600a`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:50:58.284815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-363cb773-cdc1-4994-ad30-29d1cf35ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:50:58.387788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 930/1881] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:50:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:50:58.400342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:50:58.410699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13dc3f-5e22-4458-985c-6b8569f6e376 service nova] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:50:58.414587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:50:58.415018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:50:58.415297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:50:58.415564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:50:58.425335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:50:58.426762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:50:58.441255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 952/1882] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:50:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:00.087653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762016de-97a5-497f-aa9c-eb9135f37eaf None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:00.088524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762016de-97a5-497f-aa9c-eb9135f37eaf None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:00.088818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762016de-97a5-497f-aa9c-eb9135f37eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:00.109283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-762016de-97a5-497f-aa9c-eb9135f37eaf None None] MFA Rules not processed for user `d17fb4b6648e436ab20481059f81600a`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:00.110716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-762016de-97a5-497f-aa9c-eb9135f37eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:00.133828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 931/1883] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:00.238102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:00.254684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fb69-480e-4bad-9613-3ed4ae029eeb service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:00.261898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:00.262751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:00.263166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:00.263598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] PATH_INFO: `/v3/users/a446b137625b4f90b9c4bfda2e27a19e` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:00.264894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a446b137625b4f90b9c4bfda2e27a19e)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:00.267046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:00.372636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a446b137625b4f90b9c4bfda2e27a19e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:00.372636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a446b137625b4f90b9c4bfda2e27a19e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:00.372636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a446b137625b4f90b9c4bfda2e27a19e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:00.388127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a446b137625b4f90b9c4bfda2e27a19e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:00.405409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a446b137625b4f90b9c4bfda2e27a19e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:00.461367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 953/1884] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:00 2022] DELETE /identity/v3/users/a446b137625b4f90b9c4bfda2e27a19e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:00.463431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:00.463712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:00.463982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:00.488449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:00.489898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:00.564405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 932/1885] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:00.574989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:00.577671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:00.658479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea006510-8f2b-4e1d-a7e8-d6501ceefca8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:00.663655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361311a7-c410-450b-811b-5d2dbeb8e464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:00.664774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361311a7-c410-450b-811b-5d2dbeb8e464 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:00.666160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361311a7-c410-450b-811b-5d2dbeb8e464 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:00.666660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361311a7-c410-450b-811b-5d2dbeb8e464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:00.679362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361311a7-c410-450b-811b-5d2dbeb8e464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:00.682886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361311a7-c410-450b-811b-5d2dbeb8e464 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:00.709123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 954/1886] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:00.815088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:00.817003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:00.830158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad6c30-8443-4ae0-9d0c-51b57106635b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:00.838264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:00.839215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:00.839638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:00.840135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] PATH_INFO: `/v3/users/2c0df5ebebfb4ab3b28258cc54721297` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:00.841678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0df5ebebfb4ab3b28258cc54721297)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:00.844193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:00.942219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0df5ebebfb4ab3b28258cc54721297'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:00.959825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0df5ebebfb4ab3b28258cc54721297'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:00.961731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0df5ebebfb4ab3b28258cc54721297'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.001845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0df5ebebfb4ab3b28258cc54721297'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.005696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:01.040137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0df5ebebfb4ab3b28258cc54721297'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.058616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 933/1887] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:00 2022] DELETE /identity/v3/users/2c0df5ebebfb4ab3b28258cc54721297 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:01.063738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:01.064103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:01.064430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:01.108841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:01.111153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:01.176613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:01.215617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 955/1888] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:01.232225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:01.237568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:01.323543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:01.325388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:01.354663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87456bc1-83e4-4af6-9335-8bd38ba530ab None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:01.370243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be90c-1897-41a2-88de-cfd6ae84a08c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:01.380306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:01.381037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:01.381282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:01.381583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:01.387312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:01.387964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:01.388236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:01.388526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] PATH_INFO: `/v3/projects/5547c976d8a34145b474ff65bdd2e083` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:01.391687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5547c976d8a34145b474ff65bdd2e083)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:01.393965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:01.399255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:01.401721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:01.422152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 934/1889] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:01.536096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5547c976d8a34145b474ff65bdd2e083', 'request_id': 'req-fa244e94-bc15-470b-a143-40ac66b735aa'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.551640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5547c976d8a34145b474ff65bdd2e083', 'request_id': 'req-fa244e94-bc15-470b-a143-40ac66b735aa'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.596724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5547c976d8a34145b474ff65bdd2e083', 'request_id': 'req-fa244e94-bc15-470b-a143-40ac66b735aa'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.619744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] The token cache is being invalidate because project 5547c976d8a34145b474ff65bdd2e083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:01.620508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.626692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.630058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 956/1890] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:01 2022] DELETE /identity/v3/projects/5547c976d8a34145b474ff65bdd2e083 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:01.634165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:01.650721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0ab7a-b2c9-4291-a951-b728d0f9fa46 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:01.751721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:01.752593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:01.752869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:01.753143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] PATH_INFO: `/v3/users/cc9905f9fcd74e11926165a307b6386f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:01.755327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc9905f9fcd74e11926165a307b6386f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:01.757523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:01.828186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc9905f9fcd74e11926165a307b6386f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.837206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc9905f9fcd74e11926165a307b6386f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.838385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc9905f9fcd74e11926165a307b6386f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.854941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc9905f9fcd74e11926165a307b6386f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.898257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc9905f9fcd74e11926165a307b6386f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:01.908307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 935/1891] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:01 2022] DELETE /identity/v3/users/cc9905f9fcd74e11926165a307b6386f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:02.340844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:02.346096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:02.360990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa244e94-bc15-470b-a143-40ac66b735aa admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:02.428448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:02.429940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:02.430357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:02.430778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] PATH_INFO: `/v3/projects/ac70105bdd314431a46b409a5cb19869` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:02.432620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac70105bdd314431a46b409a5cb19869)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:02.434807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:02.498424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac70105bdd314431a46b409a5cb19869', 'request_id': 'req-6f9fffa1-cc76-4934-81eb-e6187eeba42d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.507310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac70105bdd314431a46b409a5cb19869', 'request_id': 'req-6f9fffa1-cc76-4934-81eb-e6187eeba42d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.516189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac70105bdd314431a46b409a5cb19869', 'request_id': 'req-6f9fffa1-cc76-4934-81eb-e6187eeba42d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.518655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:02.521455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:02.523748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:02.535222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:02.543302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b642-6eb0-4a18-af9b-16cb6b6190ea admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:02.557870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] The token cache is being invalidate because project ac70105bdd314431a46b409a5cb19869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:02.558434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.559395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.562042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 957/1892] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:02 2022] DELETE /identity/v3/projects/ac70105bdd314431a46b409a5cb19869 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:02.565857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:02.568048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:02.633223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:02.633827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:02.634104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:02.634376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] PATH_INFO: `/v3/projects/aa57958d455841ae99e48a4141c17aad` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:02.635520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa57958d455841ae99e48a4141c17aad)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:02.637493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:02.644708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fffa1-cc76-4934-81eb-e6187eeba42d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:02.651271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:02.652194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:02.652616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:02.652888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] PATH_INFO: `/v3/users/649d37b300b7489fa89d369e35a7ab94` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:02.654028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=649d37b300b7489fa89d369e35a7ab94)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:02.656069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:02.698331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa57958d455841ae99e48a4141c17aad', 'request_id': 'req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.705142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa57958d455841ae99e48a4141c17aad', 'request_id': 'req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.713058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa57958d455841ae99e48a4141c17aad', 'request_id': 'req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.720697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '649d37b300b7489fa89d369e35a7ab94'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.721294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '649d37b300b7489fa89d369e35a7ab94'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.721954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '649d37b300b7489fa89d369e35a7ab94'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.731584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '649d37b300b7489fa89d369e35a7ab94'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.732104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] The token cache is being invalidate because project aa57958d455841ae99e48a4141c17aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:02.732418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.733165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ecf1e138-cefd-41f3-9c66-56216f7a9e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.736501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 936/1893] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:02 2022] DELETE /identity/v3/projects/aa57958d455841ae99e48a4141c17aad => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:02.742756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845fafe-36ac-4a6a-b6dd-c8b165d101b0 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:02.743556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845fafe-36ac-4a6a-b6dd-c8b165d101b0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:02.744087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845fafe-36ac-4a6a-b6dd-c8b165d101b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:02.753518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b81c030b-db0f-4d6b-a3e8-b9553a7d0260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '649d37b300b7489fa89d369e35a7ab94'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:02.780141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 958/1894] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:02 2022] DELETE /identity/v3/users/649d37b300b7489fa89d369e35a7ab94 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:02.781972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-e845fafe-36ac-4a6a-b6dd-c8b165d101b0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:02.784612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e845fafe-36ac-4a6a-b6dd-c8b165d101b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:02.881359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 937/1895] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:02.883306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721ac63-4c06-4d7e-aa7c-ff0083d286f4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:02.883586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721ac63-4c06-4d7e-aa7c-ff0083d286f4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:02.884323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721ac63-4c06-4d7e-aa7c-ff0083d286f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:02.945324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-1721ac63-4c06-4d7e-aa7c-ff0083d286f4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:02.950565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1721ac63-4c06-4d7e-aa7c-ff0083d286f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:02.968201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 959/1896] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:02.971517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845fafe-36ac-4a6a-b6dd-c8b165d101b0 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:02.983715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845fafe-36ac-4a6a-b6dd-c8b165d101b0 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:02.989527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:02.990208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:02.990745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:02.990745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:02.991388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:02.991685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:02.993534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.009196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 938/1897] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:03.013671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721ac63-4c06-4d7e-aa7c-ff0083d286f4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.026281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721ac63-4c06-4d7e-aa7c-ff0083d286f4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.031418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.031871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.032212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.033505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.034166 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.036131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.113838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 960/1898] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/projects => generated 409 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:03.114712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.129506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f544fd-0832-4a37-b84e-6056417dcf3f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.155031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:03.248623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.249513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.249807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.250073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.250960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.253311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.304111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 939/1899] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/users => generated 305 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:03.308562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.321109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cbfc1-16da-4f70-8edb-ecb846665e17 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.328013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.328570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.328920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.329301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.330725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.336350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.361984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 961/1900] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:03.369637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.392224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d1ad9-86ef-4143-96d7-c73709699928 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.393625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.396371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:03.398322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.399040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.399378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.399725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.400444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.408738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.413108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b56306-39c9-42c5-9b88-b07ea4ccdb76 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.418875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.419515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.419774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.420470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] PATH_INFO: `/v3/projects/7b2f6c1db09845538f960a83de4f9756` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.421625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2f6c1db09845538f960a83de4f9756)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.424046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.432179 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 940/1901] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:03.433700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.444537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1ad04-84c2-442f-ac97-120cca75587c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.449186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f7072-6b65-4313-8e46-b98c04ca1fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.449913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f7072-6b65-4313-8e46-b98c04ca1fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.450252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f7072-6b65-4313-8e46-b98c04ca1fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.450588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f7072-6b65-4313-8e46-b98c04ca1fcd admin admin] PATH_INFO: `/v3/projects/461b2159c1e7473887468cc2f01d5bc3/users/aaeecdcb6ee944e7ae2165141a092f99/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.463347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:INSERT=3,keystone:DELETE=12,keystone:SELECT=85 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:03.472348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f7072-6b65-4313-8e46-b98c04ca1fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=461b2159c1e7473887468cc2f01d5bc3, user_id=aaeecdcb6ee944e7ae2165141a092f99, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.475648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f7072-6b65-4313-8e46-b98c04ca1fcd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.486860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2f6c1db09845538f960a83de4f9756', 'request_id': 'req-2de9a18c-7d76-4137-bf17-0348d8c6616b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:03.491529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 941/1902] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:03 2022] PUT /identity/v3/projects/461b2159c1e7473887468cc2f01d5bc3/users/aaeecdcb6ee944e7ae2165141a092f99/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:03.494887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2f6c1db09845538f960a83de4f9756', 'request_id': 'req-2de9a18c-7d76-4137-bf17-0348d8c6616b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:03.502047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2f6c1db09845538f960a83de4f9756', 'request_id': 'req-2de9a18c-7d76-4137-bf17-0348d8c6616b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:03.505892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.505892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.505892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.521610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] The token cache is being invalidate because project 7b2f6c1db09845538f960a83de4f9756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:03.521910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:03.522677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2de9a18c-7d76-4137-bf17-0348d8c6616b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:03.526708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 962/1903] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:03 2022] DELETE /identity/v3/projects/7b2f6c1db09845538f960a83de4f9756 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:03.531255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62f131-a8a9-44c2-adce-0cb550191e3a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.531536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62f131-a8a9-44c2-adce-0cb550191e3a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.531804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62f131-a8a9-44c2-adce-0cb550191e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.550438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] MFA Rules not processed for user `aaeecdcb6ee944e7ae2165141a092f99`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:03.553005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:03.560931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9f62f131-a8a9-44c2-adce-0cb550191e3a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:03.562238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9f62f131-a8a9-44c2-adce-0cb550191e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:03.651819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 963/1904] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:03.653662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49de8-5bee-4a6d-b293-c8876489c4d0 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.654040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49de8-5bee-4a6d-b293-c8876489c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.654369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49de8-5bee-4a6d-b293-c8876489c4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.676369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 942/1905] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:03.677315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.679183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:03.683608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-1de49de8-5bee-4a6d-b293-c8876489c4d0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:03.684947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1de49de8-5bee-4a6d-b293-c8876489c4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:03.696590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f804a7-4518-4c42-850b-d9b4d66017a1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.702036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 964/1906] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:03.703078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de49de8-5bee-4a6d-b293-c8876489c4d0 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.703978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.704747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.705075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.705366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.705992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.707894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.715499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de49de8-5bee-4a6d-b293-c8876489c4d0 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.723357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.724170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.724596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.724959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.725711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:03.726131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.727920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.751269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 943/1907] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/projects => generated 409 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:03.756428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 965/1908] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:03.756583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.758321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.774103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b3888-b4dc-436d-b514-50c3e406aa45 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.827662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:03.847830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bea2a-1a01-4c29-8073-d99af882228e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.848414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.849157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.849530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.849929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.850726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.852744 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.852844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.853471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.854070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.854651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.855412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.857429 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.883525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 944/1909] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/users => generated 311 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:03.884738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 966/1910] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:03.888806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.888922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.899746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260155-bb26-479f-980f-489ae4bb86dc admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.916606 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:03.961841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.963099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.963383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.963693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.964313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.966645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.968658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab011d24-e040-420c-aa19-519b072f5373 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.974045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:03.974660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:03.974938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:03.975204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:03.975835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:03.977857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:03.978964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 945/1911] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:03.983117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:03.994275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d62203-8efd-49f1-878d-50346c49e8a8 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:03.999795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.000489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.000897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.001228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] PATH_INFO: `/v3/projects/20d9e242a746404eb8170f36759049a5/users/670e60db8126432789d6b38a94a8c095/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.004868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 967/1912] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:03 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:04.009011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.019646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d9e242a746404eb8170f36759049a5, user_id=670e60db8126432789d6b38a94a8c095, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.022390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.022561 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6520d-32dd-4a48-aae0-2cc18b3db7fe admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.027391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.028100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.028469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.028739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.029584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.031431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.045972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 946/1913] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:03 2022] PUT /identity/v3/projects/20d9e242a746404eb8170f36759049a5/users/670e60db8126432789d6b38a94a8c095/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:04.046966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.051367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 968/1914] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:04.055249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.067000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade1051-2ff6-4850-9b0a-d39ed86031fa admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.124993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efdd3c-77c3-4c9c-a2e2-bbe5e8d2943c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.130353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.131054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.131397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.131728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.132487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.134101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.140374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.140983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.141258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.141575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.142329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.144242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.144884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 947/1915] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:04.148928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.158159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 969/1916] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:04.159470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.160011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bed132-6be5-4a31-bab1-0a17464b6124 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.165170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794dafe-66bb-4cb5-ade3-706998e842b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.165980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794dafe-66bb-4cb5-ade3-706998e842b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.166457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794dafe-66bb-4cb5-ade3-706998e842b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.167004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794dafe-66bb-4cb5-ade3-706998e842b9 admin admin] PATH_INFO: `/v3/projects/20d9e242a746404eb8170f36759049a5/users/670e60db8126432789d6b38a94a8c095/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.169626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794dafe-66bb-4cb5-ade3-706998e842b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d9e242a746404eb8170f36759049a5, user_id=670e60db8126432789d6b38a94a8c095, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.172574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794dafe-66bb-4cb5-ade3-706998e842b9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.173097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9f58-9762-4551-bdb5-246870ae8d03 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.178925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b5169-37d3-4184-ab43-617125b560db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.179593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b5169-37d3-4184-ab43-617125b560db admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.179869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b5169-37d3-4184-ab43-617125b560db admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.180143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b5169-37d3-4184-ab43-617125b560db admin admin] PATH_INFO: `/v3/projects/6f3ed347f81b4b8eab8a3f4deee42be1/users/ca89d28b5dac4364bf921543775ceff7/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.183907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 948/1917] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:04 2022] PUT /identity/v3/projects/20d9e242a746404eb8170f36759049a5/users/670e60db8126432789d6b38a94a8c095/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:04.195231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.195956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.196439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.198395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b5169-37d3-4184-ab43-617125b560db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3ed347f81b4b8eab8a3f4deee42be1, user_id=ca89d28b5dac4364bf921543775ceff7, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.201312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b5169-37d3-4184-ab43-617125b560db admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.217541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 970/1918] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:04 2022] PUT /identity/v3/projects/6f3ed347f81b4b8eab8a3f4deee42be1/users/ca89d28b5dac4364bf921543775ceff7/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:04.227963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] MFA Rules not processed for user `670e60db8126432789d6b38a94a8c095`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:04.229313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:04.229490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feaa1f-a39b-4e9c-927c-83ce41e58164 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.229783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feaa1f-a39b-4e9c-927c-83ce41e58164 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.230056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feaa1f-a39b-4e9c-927c-83ce41e58164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.247213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:04.260782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d1feaa1f-a39b-4e9c-927c-83ce41e58164 None None] MFA Rules not processed for user `ca89d28b5dac4364bf921543775ceff7`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:04.262007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d1feaa1f-a39b-4e9c-927c-83ce41e58164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:04.309566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 949/1919] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:04.321855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.324029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:04.338758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:04.344869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 971/1920] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:04.350953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feaa1f-a39b-4e9c-927c-83ce41e58164 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.395572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df1eb6-dffa-4305-91f2-1a968c656615 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.399220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.399897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.400234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.400606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.411320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.413045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.416479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feaa1f-a39b-4e9c-927c-83ce41e58164 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.422540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.423213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.423686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.423961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.424624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.424726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 950/1921] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:04 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:04.426534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.468858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 972/1922] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:04 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:04.470613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.485018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929386a0-136a-4169-9274-9afe0c80d0bc service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.553769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.554470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.554785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.555190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.555848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.557945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.610455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 951/1923] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:04 2022] POST /identity/v3/users => generated 298 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:04.610956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.611855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.622035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4821da5-5e25-407a-977d-b77c75bab374 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.627907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.628659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.629079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.629444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.630186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.633482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.645532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 952/1924] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:04.649422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.661714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85814481-4471-4321-8454-d33c8708ff2a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.666859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.667634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.667908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.668171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] PATH_INFO: `/v3/projects/29ebffe162894a348f581c9666199618/users/0f9d4cfa614441b099e063e55bea0574/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.684144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074efb3b-800a-4c4e-aca3-7e823f13672a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.686351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ebffe162894a348f581c9666199618, user_id=0f9d4cfa614441b099e063e55bea0574, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.689052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.689191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.689917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.690218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.690557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.704524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 953/1925] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:04 2022] PUT /identity/v3/projects/29ebffe162894a348f581c9666199618/users/0f9d4cfa614441b099e063e55bea0574/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:04.709017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.722648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:04.765105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.767867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.780843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 973/1926] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:04 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:51:04.799129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9544e3-001a-4a5e-a26e-84e7151ff0e4 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.804008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819026b-e4ce-4be9-a2da-a9763a041cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.804769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819026b-e4ce-4be9-a2da-a9763a041cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.805370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819026b-e4ce-4be9-a2da-a9763a041cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.805633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819026b-e4ce-4be9-a2da-a9763a041cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.806289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819026b-e4ce-4be9-a2da-a9763a041cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.807904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819026b-e4ce-4be9-a2da-a9763a041cb0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.819832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 954/1927] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:04.823762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:04.839330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c74b76-c84d-49b8-a665-b75c3d8bdf1d service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:04.844240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:04.845337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.845822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.846244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] PATH_INFO: `/v3/projects/29ebffe162894a348f581c9666199618/users/0f9d4cfa614441b099e063e55bea0574/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.848504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ebffe162894a348f581c9666199618, user_id=0f9d4cfa614441b099e063e55bea0574, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:04.852501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:04.876460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 974/1928] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:04 2022] PUT /identity/v3/projects/29ebffe162894a348f581c9666199618/users/0f9d4cfa614441b099e063e55bea0574/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:04.878479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c56a2-f52f-4a52-848a-b528e80d202d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:04.878766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c56a2-f52f-4a52-848a-b528e80d202d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:04.879044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c56a2-f52f-4a52-848a-b528e80d202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:04.924629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4f8c56a2-f52f-4a52-848a-b528e80d202d None None] MFA Rules not processed for user `0f9d4cfa614441b099e063e55bea0574`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:04.926032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4f8c56a2-f52f-4a52-848a-b528e80d202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:05.016426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 955/1929] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:05.032807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:05.035833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:05.124580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:05.189771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1d448-89dd-4cbe-9cd8-8c79231497e4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:05.203671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:05.204392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:05.204700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:05.204955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:05.327448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:05.329314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:05.357540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 975/1930] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:12.248178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c56a2-f52f-4a52-848a-b528e80d202d None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:12.260187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c56a2-f52f-4a52-848a-b528e80d202d None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:12.265694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e889fb5-520b-48b9-a1ea-0b43317a43e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:12.268008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e889fb5-520b-48b9-a1ea-0b43317a43e0 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:12.268875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e889fb5-520b-48b9-a1ea-0b43317a43e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:12.269356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e889fb5-520b-48b9-a1ea-0b43317a43e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:12.285926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e889fb5-520b-48b9-a1ea-0b43317a43e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:12.287843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e889fb5-520b-48b9-a1ea-0b43317a43e0 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:12.309279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 956/1931] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:12.798688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:12.869376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d3bb3-f912-4e2f-88f8-44f717971794 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:12.874082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:12.874807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:12.875152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:12.875491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] PATH_INFO: `/v3/users/aaeecdcb6ee944e7ae2165141a092f99` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:12.876899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaeecdcb6ee944e7ae2165141a092f99)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:12.879552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:12.940771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaeecdcb6ee944e7ae2165141a092f99'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:12.941683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaeecdcb6ee944e7ae2165141a092f99'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:12.942425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaeecdcb6ee944e7ae2165141a092f99'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:12.962990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaeecdcb6ee944e7ae2165141a092f99'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:12.967903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaeecdcb6ee944e7ae2165141a092f99'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:12.981789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:12.986486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 976/1932] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:12 2022] DELETE /identity/v3/users/aaeecdcb6ee944e7ae2165141a092f99 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:12.991335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:12.991732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:12.991879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:13.011339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:13.012688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:13.073769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 957/1933] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:13.083468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:13.085239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:13.162272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35119059-e28c-4754-9112-56f5696be490 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:13.166864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:13.167582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:13.167915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:13.168292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:13.189012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:13.190705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:13.203328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 977/1934] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:13.589930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:13.592824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:13.610633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57251212-129b-436d-82ac-f5cef6a52c70 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:13.615519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:13.616158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:13.616631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:13.616898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] PATH_INFO: `/v3/projects/461b2159c1e7473887468cc2f01d5bc3` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:13.618154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=461b2159c1e7473887468cc2f01d5bc3)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:13.620063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:13.653812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:13.706542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461b2159c1e7473887468cc2f01d5bc3', 'request_id': 'req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.713466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461b2159c1e7473887468cc2f01d5bc3', 'request_id': 'req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.720644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461b2159c1e7473887468cc2f01d5bc3', 'request_id': 'req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.750100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] The token cache is being invalidate because project 461b2159c1e7473887468cc2f01d5bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:13.750888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.751700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.754033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 958/1935] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:13 2022] DELETE /identity/v3/projects/461b2159c1e7473887468cc2f01d5bc3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:13.757924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:13.770919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb65083-ea1d-420f-8a19-1428899554ba service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:13.846206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:13.846854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:13.847113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:13.847378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] PATH_INFO: `/v3/users/670e60db8126432789d6b38a94a8c095` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:13.849143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=670e60db8126432789d6b38a94a8c095)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:13.850708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:13.923228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670e60db8126432789d6b38a94a8c095'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.924773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670e60db8126432789d6b38a94a8c095'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.924773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670e60db8126432789d6b38a94a8c095'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.935066 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670e60db8126432789d6b38a94a8c095'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.957014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e56f247-809d-471d-a6fc-68179289c7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670e60db8126432789d6b38a94a8c095'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:13.985512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 978/1936] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:13 2022] DELETE /identity/v3/users/670e60db8126432789d6b38a94a8c095 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:14.394139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:14.395615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:14.424004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ffe7b-3173-4e4f-ade0-98359c1b6fd3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:14.539612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:14.539849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:14.539938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:14.540212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] PATH_INFO: `/v3/projects/20d9e242a746404eb8170f36759049a5` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:14.542270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20d9e242a746404eb8170f36759049a5)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:14.544836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:14.640771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20d9e242a746404eb8170f36759049a5', 'request_id': 'req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:14.653291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20d9e242a746404eb8170f36759049a5', 'request_id': 'req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:14.677692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20d9e242a746404eb8170f36759049a5', 'request_id': 'req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:14.703681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] The token cache is being invalidate because project 20d9e242a746404eb8170f36759049a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:14.704384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:14.707250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6802d7ef-edc2-4ab0-b2c6-90ca321b3b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:14.709825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 959/1937] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:14 2022] DELETE /identity/v3/projects/20d9e242a746404eb8170f36759049a5 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:14.714959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddee0d-ba62-4376-be8c-a9cb22a5cbb6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:14.715405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddee0d-ba62-4376-be8c-a9cb22a5cbb6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:14.715820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddee0d-ba62-4376-be8c-a9cb22a5cbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:14.756956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0bddee0d-ba62-4376-be8c-a9cb22a5cbb6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:14.758452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0bddee0d-ba62-4376-be8c-a9cb22a5cbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:14.819552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:14.889792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 979/1938] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:14.898096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52ea3-3156-477c-86e7-a2f645001f26 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:14.898585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52ea3-3156-477c-86e7-a2f645001f26 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:14.899055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52ea3-3156-477c-86e7-a2f645001f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:14.925010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-05b52ea3-3156-477c-86e7-a2f645001f26 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:14.926563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-05b52ea3-3156-477c-86e7-a2f645001f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:14.938984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 960/1939] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:14.942998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddee0d-ba62-4376-be8c-a9cb22a5cbb6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:14.954924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddee0d-ba62-4376-be8c-a9cb22a5cbb6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:14.960566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:14.961371 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:14.961713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:14.962169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:14.962890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:14.963252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:14.965157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.005459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 980/1940] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:15.005629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52ea3-3156-477c-86e7-a2f645001f26 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.017537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52ea3-3156-477c-86e7-a2f645001f26 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.027119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.027787 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.028065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.030799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.036648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.039438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.091364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 961/1941] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:15 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:15.095287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.117402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716094c-c771-4ff9-9b0e-3b847aaec851 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.239981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.241197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.241933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.242435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.243280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.246876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.289373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 981/1942] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:15 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:15.294259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.302670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:15.306676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ac83-f67f-4dda-8268-cb7969b4748c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.315005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.315688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.316188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.316681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.317633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.319397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.332134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 962/1943] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:15.336460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.349042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d68847-1df3-4b97-b01b-2a34e128a395 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.354565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725220d-6a90-4e03-b4b1-9b2756828b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.355814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725220d-6a90-4e03-b4b1-9b2756828b2a admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.356097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725220d-6a90-4e03-b4b1-9b2756828b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.356526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725220d-6a90-4e03-b4b1-9b2756828b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.357256 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725220d-6a90-4e03-b4b1-9b2756828b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.359207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725220d-6a90-4e03-b4b1-9b2756828b2a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.376320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 982/1944] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:15.379373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.390378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c8191-cce8-4b0c-bc8d-8f8985959d3a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.397085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.397879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.398181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.398421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] PATH_INFO: `/v3/projects/6accd591b5c14589bd942ed082e50eed/users/59c7187e3a5d42dd90180520fab2f21a/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.419362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6accd591b5c14589bd942ed082e50eed, user_id=59c7187e3a5d42dd90180520fab2f21a, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.422469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.452827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 963/1945] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:15 2022] PUT /identity/v3/projects/6accd591b5c14589bd942ed082e50eed/users/59c7187e3a5d42dd90180520fab2f21a/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:15.465411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e2c8f-fbf6-4014-ba14-ee19e97a4130 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.465698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e2c8f-fbf6-4014-ba14-ee19e97a4130 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.465968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e2c8f-fbf6-4014-ba14-ee19e97a4130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.501714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c52e2c8f-fbf6-4014-ba14-ee19e97a4130 None None] MFA Rules not processed for user `59c7187e3a5d42dd90180520fab2f21a`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:15.503155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c52e2c8f-fbf6-4014-ba14-ee19e97a4130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:15.577521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 983/1946] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:15.582129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.652653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0465c4-4d09-4448-9fd1-61f8941e5c84 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.657544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.658148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.658413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.658678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.659301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.661724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.696157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 964/1947] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:15 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:15.700489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e2c8f-fbf6-4014-ba14-ee19e97a4130 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.713740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e2c8f-fbf6-4014-ba14-ee19e97a4130 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.729105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:15.777578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.778201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.778471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.778735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.779357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.781444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.805290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 984/1948] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:15 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:15.810692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.834306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949e890-5f03-4e5b-ac2d-d4592644ec1f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.848993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.850082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.850191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.850459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.851118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.854784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.875316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 965/1949] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:15.880655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.891587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d2911-a855-4a4c-95aa-d0005d522113 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.898307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fe413-7de0-4c62-bbac-04957c4803e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.899697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe413-7de0-4c62-bbac-04957c4803e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.899697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe413-7de0-4c62-bbac-04957c4803e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.899697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe413-7de0-4c62-bbac-04957c4803e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.900653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fe413-7de0-4c62-bbac-04957c4803e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.903199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fe413-7de0-4c62-bbac-04957c4803e4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.917014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 985/1950] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:15.921706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:15.934601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31466b16-2535-41a2-b6d9-1d933aba487e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:15.939957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:15.940662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.940935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.941207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] PATH_INFO: `/v3/projects/14be55c8383e4294aa3671831d56fb92/users/79870e0efbb94ac28bbb601ba6c1e2d1/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:15.960431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14be55c8383e4294aa3671831d56fb92, user_id=79870e0efbb94ac28bbb601ba6c1e2d1, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:15.963253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:15.978418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 966/1951] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:15 2022] PUT /identity/v3/projects/14be55c8383e4294aa3671831d56fb92/users/79870e0efbb94ac28bbb601ba6c1e2d1/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:15.989116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efbe94-7317-42f0-9366-5b1639c60775 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:15.989487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efbe94-7317-42f0-9366-5b1639c60775 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:15.990500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efbe94-7317-42f0-9366-5b1639c60775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.021363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-55efbe94-7317-42f0-9366-5b1639c60775 None None] MFA Rules not processed for user `79870e0efbb94ac28bbb601ba6c1e2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:16.022587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-55efbe94-7317-42f0-9366-5b1639c60775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:16.120400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 986/1952] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:16.123092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.211742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748426f-1f91-4192-948d-7cae93140b74 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.217888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.219067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.219067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.219067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.219694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:16.222737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:16.267623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 967/1953] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:16 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:16.269151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efbe94-7317-42f0-9366-5b1639c60775 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.291633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efbe94-7317-42f0-9366-5b1639c60775 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.362760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.363589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.364003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.364449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.365435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:16.370751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:16.396604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 987/1954] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:16 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:16.397853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.404474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:16.409097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716768-08b3-4440-9cf3-270fbd35b19b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.416075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.416967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.418117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.418117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.419063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:16.431712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:16.443679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 968/1955] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:16.447926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.458301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d44bd-75fb-46ef-803a-8691dde5ad62 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.463018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.463693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.463964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.464231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] PATH_INFO: `/v3/projects/131cd28197264ce3bb6d101e49b612f6/users/19b2e18e739d4b17ade11bd139154965/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.478297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:16.481101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=131cd28197264ce3bb6d101e49b612f6, user_id=19b2e18e739d4b17ade11bd139154965, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:16.483763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:16.499070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 988/1956] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:16 2022] PUT /identity/v3/projects/131cd28197264ce3bb6d101e49b612f6/users/19b2e18e739d4b17ade11bd139154965/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:16.504315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.518814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d80363-0485-4f14-95ca-9397555285c6 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.583435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01be76-f956-4d4d-81c2-ae76b9344b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.584037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01be76-f956-4d4d-81c2-ae76b9344b06 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.584337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01be76-f956-4d4d-81c2-ae76b9344b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.584607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01be76-f956-4d4d-81c2-ae76b9344b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.585240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01be76-f956-4d4d-81c2-ae76b9344b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:16.586827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01be76-f956-4d4d-81c2-ae76b9344b06 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:16.598088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 969/1957] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:16.602119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.612383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4df72-3294-46f4-ac8d-908392ef8393 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.617118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.617778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.618051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.618320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] PATH_INFO: `/v3/projects/131cd28197264ce3bb6d101e49b612f6/users/19b2e18e739d4b17ade11bd139154965/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.620286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=131cd28197264ce3bb6d101e49b612f6, user_id=19b2e18e739d4b17ade11bd139154965, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:16.622948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:16.635182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 989/1958] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:16 2022] PUT /identity/v3/projects/131cd28197264ce3bb6d101e49b612f6/users/19b2e18e739d4b17ade11bd139154965/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:16.647136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141377e-3b1f-4530-8cdc-52aeae47847e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.647374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141377e-3b1f-4530-8cdc-52aeae47847e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.647680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141377e-3b1f-4530-8cdc-52aeae47847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.677838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7141377e-3b1f-4530-8cdc-52aeae47847e None None] MFA Rules not processed for user `19b2e18e739d4b17ade11bd139154965`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:16.679089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7141377e-3b1f-4530-8cdc-52aeae47847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:16.771641 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 970/1959] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:16.779669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.782345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:16.846212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdca20-ff5f-4214-bb32-160ba3d56256 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.850014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.850703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.851243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.851466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:16.861518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:16.863007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:16.874575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 990/1960] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:16.926285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141377e-3b1f-4530-8cdc-52aeae47847e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:16.962433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141377e-3b1f-4530-8cdc-52aeae47847e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:16.967587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ed3ea-4935-45bf-b945-56a9a043d822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:16.968207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ed3ea-4935-45bf-b945-56a9a043d822 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:16.968676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ed3ea-4935-45bf-b945-56a9a043d822 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:16.968941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ed3ea-4935-45bf-b945-56a9a043d822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:17.046902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ed3ea-4935-45bf-b945-56a9a043d822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:17.049040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ed3ea-4935-45bf-b945-56a9a043d822 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:17.075490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 971/1961] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:51:17.464895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:17.466440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:17.641164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb5888-2ac6-467f-9a2f-4b44277b092f service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:17.674169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:17.675958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:17.676381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:17.676712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] PATH_INFO: `/v3/users/ca89d28b5dac4364bf921543775ceff7` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:17.678237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca89d28b5dac4364bf921543775ceff7)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:17.680215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:17.757268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca89d28b5dac4364bf921543775ceff7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:17.758312 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca89d28b5dac4364bf921543775ceff7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:17.759536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca89d28b5dac4364bf921543775ceff7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:17.773886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca89d28b5dac4364bf921543775ceff7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:17.790028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca89d28b5dac4364bf921543775ceff7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:17.813726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:17.819068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 991/1962] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:17 2022] DELETE /identity/v3/users/ca89d28b5dac4364bf921543775ceff7 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:17.823598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:17.824074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:17.828696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:17.849904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:17.851241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:17.862475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:17.923021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 972/1963] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:17.936598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:17.938616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:18.006743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f305b3-41b1-49a0-a866-244c43153c0d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:18.010964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:18.011593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:18.012341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:18.012627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:18.027933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:18.029410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:18.040083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 992/1964] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:18.337368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:18.349836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:18.362761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd3da6-7fc1-446e-aa75-ba55ec9ee457 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:18.368313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:18.369067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:18.369347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:18.369615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] PATH_INFO: `/v3/projects/6f3ed347f81b4b8eab8a3f4deee42be1` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:18.371097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3ed347f81b4b8eab8a3f4deee42be1)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:18.373209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:18.430209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3ed347f81b4b8eab8a3f4deee42be1', 'request_id': 'req-57ad89a3-38a5-4d86-8009-92513fea3209'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.438604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3ed347f81b4b8eab8a3f4deee42be1', 'request_id': 'req-57ad89a3-38a5-4d86-8009-92513fea3209'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.446049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3ed347f81b4b8eab8a3f4deee42be1', 'request_id': 'req-57ad89a3-38a5-4d86-8009-92513fea3209'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.461015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] The token cache is being invalidate because project 6f3ed347f81b4b8eab8a3f4deee42be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:18.461298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.461921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.463446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 973/1965] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:18 2022] DELETE /identity/v3/projects/6f3ed347f81b4b8eab8a3f4deee42be1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:18.467768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:18.482382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a98cc5-296e-460e-877d-1ba5fe202f71 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:18.544396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:18.544960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:18.545232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:18.545501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] PATH_INFO: `/v3/users/0f9d4cfa614441b099e063e55bea0574` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:18.546748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9d4cfa614441b099e063e55bea0574)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:18.548927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:18.611659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9d4cfa614441b099e063e55bea0574'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.612376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9d4cfa614441b099e063e55bea0574'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.612376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9d4cfa614441b099e063e55bea0574'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.623882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9d4cfa614441b099e063e55bea0574'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.635431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d61a8f81-7c2e-4d39-b685-6f44f27e7d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9d4cfa614441b099e063e55bea0574'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:18.656081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 993/1966] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:18 2022] DELETE /identity/v3/users/0f9d4cfa614441b099e063e55bea0574 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:18.959180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:18.960625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:18.976184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad89a3-38a5-4d86-8009-92513fea3209 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.070117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.070986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.071388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.072889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] PATH_INFO: `/v3/projects/29ebffe162894a348f581c9666199618` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.074036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ebffe162894a348f581c9666199618)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.076207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.137523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:19.149062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ebffe162894a348f581c9666199618', 'request_id': 'req-67c99357-32cf-49af-90a4-f13b35ced18c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:19.158066 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ebffe162894a348f581c9666199618', 'request_id': 'req-67c99357-32cf-49af-90a4-f13b35ced18c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:19.164784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ebffe162894a348f581c9666199618', 'request_id': 'req-67c99357-32cf-49af-90a4-f13b35ced18c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:19.183873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] The token cache is being invalidate because project 29ebffe162894a348f581c9666199618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:19.184160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:19.185610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67c99357-32cf-49af-90a4-f13b35ced18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:19.187797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 974/1967] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:18 2022] DELETE /identity/v3/projects/29ebffe162894a348f581c9666199618 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:19.193687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a1be9-61da-4b81-b5c8-1a0008b9cdea None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.193968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a1be9-61da-4b81-b5c8-1a0008b9cdea None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.194241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a1be9-61da-4b81-b5c8-1a0008b9cdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.230872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-903a1be9-61da-4b81-b5c8-1a0008b9cdea None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:19.232081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-903a1be9-61da-4b81-b5c8-1a0008b9cdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:19.250662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:19.305350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 994/1968] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:19.307532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65831fa-7e52-44bc-b057-f9f629527701 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.307990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65831fa-7e52-44bc-b057-f9f629527701 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.308319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65831fa-7e52-44bc-b057-f9f629527701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.331202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a65831fa-7e52-44bc-b057-f9f629527701 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:19.332620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a65831fa-7e52-44bc-b057-f9f629527701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:19.340552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 975/1969] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:19.344702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a1be9-61da-4b81-b5c8-1a0008b9cdea None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:19.354499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a1be9-61da-4b81-b5c8-1a0008b9cdea None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.359035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.359632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.359905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.360250 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.360915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:19.361207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.362802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.374727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 995/1970] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:19.381050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65831fa-7e52-44bc-b057-f9f629527701 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:19.390577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65831fa-7e52-44bc-b057-f9f629527701 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.395783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.396421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.396695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.396961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.397595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.399504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.429293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 976/1971] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:19 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:19.433685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:19.444511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b63b78-d50f-44d7-9e36-ae8d90a930a4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.510377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.511075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.511295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.511564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.512215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.514252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.543010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 996/1972] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:19 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:19.547324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:19.562698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28075ea-b9c5-43ca-90e6-2751d4f15921 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.572676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.573310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.573585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.573853 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.574564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.576161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.587441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 977/1973] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:19.591567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:19.603098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3db02-1481-4b7a-89b2-4d0c08d6582e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.608587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387bc4-a17c-4a2f-91cb-816706d66a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.609191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387bc4-a17c-4a2f-91cb-816706d66a45 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.609460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387bc4-a17c-4a2f-91cb-816706d66a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.609723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387bc4-a17c-4a2f-91cb-816706d66a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.610379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387bc4-a17c-4a2f-91cb-816706d66a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.612042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387bc4-a17c-4a2f-91cb-816706d66a45 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.623403 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 997/1974] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:19.628198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:19.638350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a966ea-6819-482f-834a-13a8df76d9c7 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.643145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.644230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.644609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.644944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] PATH_INFO: `/v3/projects/5a5af40474ef4ed09d723d746a56c5fd/users/c9e4d103e1af4d07af4ea5c2fb1e51f3/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.660357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5af40474ef4ed09d723d746a56c5fd, user_id=c9e4d103e1af4d07af4ea5c2fb1e51f3, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.663104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.679896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 978/1975] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:19 2022] PUT /identity/v3/projects/5a5af40474ef4ed09d723d746a56c5fd/users/c9e4d103e1af4d07af4ea5c2fb1e51f3/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:19.691181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.691486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.691757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.720412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] MFA Rules not processed for user `c9e4d103e1af4d07af4ea5c2fb1e51f3`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:19.721631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:19.789323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 998/1976] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:19.800687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:19.802543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:19.862626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12e893-55cb-45e5-a8d7-3343447cdcc1 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:19.866338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7ba96-d33b-41ed-802f-9d39a7287946 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8571f3d80b0749729012353fcbbd9e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:19.867212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7ba96-d33b-41ed-802f-9d39a7287946 service glance] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:19.867539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7ba96-d33b-41ed-802f-9d39a7287946 service glance] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:19.868120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7ba96-d33b-41ed-802f-9d39a7287946 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:19.878056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7ba96-d33b-41ed-802f-9d39a7287946 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:19.879604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7ba96-d33b-41ed-802f-9d39a7287946 service glance] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:19.890917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 979/1977] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:20.338648 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:20.349839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:20.376708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:20.426954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f45a8-66ab-4add-b54e-c39064efec2d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:20.431402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:20.432033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:20.432325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:20.432592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] PATH_INFO: `/v3/users/59c7187e3a5d42dd90180520fab2f21a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:20.433695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c7187e3a5d42dd90180520fab2f21a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:20.435613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:20.484667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c7187e3a5d42dd90180520fab2f21a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:20.485344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c7187e3a5d42dd90180520fab2f21a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:20.485963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c7187e3a5d42dd90180520fab2f21a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:20.492900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c7187e3a5d42dd90180520fab2f21a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:20.502670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c7187e3a5d42dd90180520fab2f21a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:20.520321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 999/1978] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:20 2022] DELETE /identity/v3/users/59c7187e3a5d42dd90180520fab2f21a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:20.525115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:20.525582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:20.525680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:20.544752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:20.545935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:20.617846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 980/1979] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:20.628028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:20.629794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:20.689480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a2d90-8745-4319-ac85-cf8b5f6ad475 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:20.693350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:20.693984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:20.694244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:20.694516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:20.705033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:20.707377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:20.721089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1000/1980] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:51:21.059199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:21.060673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:21.075013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b174f74-9829-4abf-86e3-db0303aa04f1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:21.083257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:21.083742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:21.084017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:21.084513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] PATH_INFO: `/v3/projects/6accd591b5c14589bd942ed082e50eed` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:21.085711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6accd591b5c14589bd942ed082e50eed)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:21.087650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:21.092445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:21.157697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6accd591b5c14589bd942ed082e50eed', 'request_id': 'req-0670432d-620d-484e-b775-1180a8a4d85b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.163286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6accd591b5c14589bd942ed082e50eed', 'request_id': 'req-0670432d-620d-484e-b775-1180a8a4d85b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.169586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6accd591b5c14589bd942ed082e50eed', 'request_id': 'req-0670432d-620d-484e-b775-1180a8a4d85b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.183469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] The token cache is being invalidate because project 6accd591b5c14589bd942ed082e50eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:21.183751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.184394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.185955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 981/1981] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:21 2022] DELETE /identity/v3/projects/6accd591b5c14589bd942ed082e50eed => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:21.190645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:21.201710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09993-e111-4ce2-b7c6-4e3bec8f8381 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:21.246162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:21.277180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:21.277920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:21.278276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:21.278682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] PATH_INFO: `/v3/users/79870e0efbb94ac28bbb601ba6c1e2d1` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:21.281228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=79870e0efbb94ac28bbb601ba6c1e2d1)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:21.284109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:21.340490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79870e0efbb94ac28bbb601ba6c1e2d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.341508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79870e0efbb94ac28bbb601ba6c1e2d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.343109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79870e0efbb94ac28bbb601ba6c1e2d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.350370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79870e0efbb94ac28bbb601ba6c1e2d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.358180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79870e0efbb94ac28bbb601ba6c1e2d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.380767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1001/1982] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:21 2022] DELETE /identity/v3/users/79870e0efbb94ac28bbb601ba6c1e2d1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:21.687266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:21.688891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:21.706979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670432d-620d-484e-b775-1180a8a4d85b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:21.777546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:21.778174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:21.778445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:21.778709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] PATH_INFO: `/v3/projects/14be55c8383e4294aa3671831d56fb92` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:21.779808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] RBAC: Authorizing `identity:delete_project(project_id=14be55c8383e4294aa3671831d56fb92)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:21.781738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:21.831015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14be55c8383e4294aa3671831d56fb92', 'request_id': 'req-106d10f3-8209-4d0f-89ba-14fd51c9088c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.836842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14be55c8383e4294aa3671831d56fb92', 'request_id': 'req-106d10f3-8209-4d0f-89ba-14fd51c9088c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.843256 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14be55c8383e4294aa3671831d56fb92', 'request_id': 'req-106d10f3-8209-4d0f-89ba-14fd51c9088c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.861856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] The token cache is being invalidate because project 14be55c8383e4294aa3671831d56fb92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:21.862125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.862748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:21.864744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 982/1983] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:21 2022] DELETE /identity/v3/projects/14be55c8383e4294aa3671831d56fb92 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:21.869054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:21.870528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:21.886308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbdf8c-56cc-40f7-91d4-1465e24ef07a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:21.972293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:21.972924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:21.973197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:21.973635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] PATH_INFO: `/v3/users/19b2e18e739d4b17ade11bd139154965` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:21.975470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b2e18e739d4b17ade11bd139154965)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:21.979277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:22.047276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b2e18e739d4b17ade11bd139154965'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.048072 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b2e18e739d4b17ade11bd139154965'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.048916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b2e18e739d4b17ade11bd139154965'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.056733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b2e18e739d4b17ade11bd139154965'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.060695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:22.068177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-852fd767-2562-4aaf-bef5-b7a927579fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b2e18e739d4b17ade11bd139154965'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.088004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1002/1984] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:21 2022] DELETE /identity/v3/users/19b2e18e739d4b17ade11bd139154965 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:22.727663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:22.729225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:22.744663 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d10f3-8209-4d0f-89ba-14fd51c9088c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:22.766084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:22.817644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:22.818284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:22.818723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:22.819189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] PATH_INFO: `/v3/projects/131cd28197264ce3bb6d101e49b612f6` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:22.820479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=131cd28197264ce3bb6d101e49b612f6)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:22.822484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:22.889156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131cd28197264ce3bb6d101e49b612f6', 'request_id': 'req-84206af8-e6b8-4a76-b647-e06d2201c338'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.896702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131cd28197264ce3bb6d101e49b612f6', 'request_id': 'req-84206af8-e6b8-4a76-b647-e06d2201c338'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.903304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131cd28197264ce3bb6d101e49b612f6', 'request_id': 'req-84206af8-e6b8-4a76-b647-e06d2201c338'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.924399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] The token cache is being invalidate because project 131cd28197264ce3bb6d101e49b612f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:22.924685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.925333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-84206af8-e6b8-4a76-b647-e06d2201c338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:22.926912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 983/1985] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:22 2022] DELETE /identity/v3/projects/131cd28197264ce3bb6d101e49b612f6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:22.935151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be198b-9b36-47a7-8d71-468f4a1019a6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:22.935151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be198b-9b36-47a7-8d71-468f4a1019a6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:22.935151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be198b-9b36-47a7-8d71-468f4a1019a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:22.954968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-41be198b-9b36-47a7-8d71-468f4a1019a6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:22.956225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-41be198b-9b36-47a7-8d71-468f4a1019a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:23.019651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1003/1986] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:23.027917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78845b6a-24db-4949-aae7-6572940d8ba2 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.028307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78845b6a-24db-4949-aae7-6572940d8ba2 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.028571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78845b6a-24db-4949-aae7-6572940d8ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.051249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-78845b6a-24db-4949-aae7-6572940d8ba2 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:23.052718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-78845b6a-24db-4949-aae7-6572940d8ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:23.062668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 984/1987] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:23.067393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be198b-9b36-47a7-8d71-468f4a1019a6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.077409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be198b-9b36-47a7-8d71-468f4a1019a6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.085942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.086566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.086869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.087138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.087795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:23.088104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.089750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.108391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78845b6a-24db-4949-aae7-6572940d8ba2 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.108627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1004/1988] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:23.125526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78845b6a-24db-4949-aae7-6572940d8ba2 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.131676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.132400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.132677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.132950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.133628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.135826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.165814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 985/1989] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:23 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:23.170070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.182064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a951f-4bc5-4b2d-941c-f4d9cbbd5a15 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.243587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.244197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.244565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.244771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.245397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.247354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.272284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1005/1990] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:23 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:23.276594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.287372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08dd52-6438-4e95-9e44-ef9b447412d3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.291996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.292637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.292905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.293166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.293798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.295901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.311559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 986/1991] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:23.317253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.327502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dc6e0-d522-4cb3-b30b-1b0e485d463e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.336902 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acea08-139a-4603-8132-b0097ea7c6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.337566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acea08-139a-4603-8132-b0097ea7c6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.337783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acea08-139a-4603-8132-b0097ea7c6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.338045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acea08-139a-4603-8132-b0097ea7c6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.338688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acea08-139a-4603-8132-b0097ea7c6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.340376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acea08-139a-4603-8132-b0097ea7c6a2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.356347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1006/1992] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:23.362705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.373214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d878a-a340-4345-a4d5-4e138fa727ac admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.378041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.378694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.378966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.379236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] PATH_INFO: `/v3/projects/b6d9faed9db344c4a7ef3d87273a4c07/users/9df8e6679cb84d21bc50ad69f646990f/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.394441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d9faed9db344c4a7ef3d87273a4c07, user_id=9df8e6679cb84d21bc50ad69f646990f, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.397147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.411491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 987/1993] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:23 2022] PUT /identity/v3/projects/b6d9faed9db344c4a7ef3d87273a4c07/users/9df8e6679cb84d21bc50ad69f646990f/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:23.423143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7442f5d-1935-40fb-9087-a8350499f2a7 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.423433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7442f5d-1935-40fb-9087-a8350499f2a7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.423741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7442f5d-1935-40fb-9087-a8350499f2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.451388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e7442f5d-1935-40fb-9087-a8350499f2a7 None None] MFA Rules not processed for user `9df8e6679cb84d21bc50ad69f646990f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:23.452693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e7442f5d-1935-40fb-9087-a8350499f2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:23.487335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:23.524015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1007/1994] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:23.529731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.589032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:23.607625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162812-8f9d-40f9-a250-7b8dac7cdb3c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.613547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.614209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.614485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.614750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.615376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.617389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.646916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 988/1995] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:23 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:23.651234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7442f5d-1935-40fb-9087-a8350499f2a7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.683453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7442f5d-1935-40fb-9087-a8350499f2a7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.741676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.742330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.742626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.742893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.743544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.745536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.775212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1008/1996] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:23 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:23.779527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.790948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c32c4-e086-41f3-a20a-c9949096d72c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.795671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.796297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.796586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.796855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.797500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.799164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.810252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 989/1997] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:23.815492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.825702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e6c8f-8a8a-4928-86b3-886bafde8a52 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.830868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.831635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.831915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.832180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] PATH_INFO: `/v3/projects/3bc32b9cf64d43faa7612e57a4c12357/users/847cbf239cbb424abc3a7294dd1a05b8/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.846917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc32b9cf64d43faa7612e57a4c12357, user_id=847cbf239cbb424abc3a7294dd1a05b8, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.849602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.866443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1009/1998] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:23 2022] PUT /identity/v3/projects/3bc32b9cf64d43faa7612e57a4c12357/users/847cbf239cbb424abc3a7294dd1a05b8/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:23.870460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.881363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd2ffc-7bb2-4a1b-88dd-297ad8fec02a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.952170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca368e4-9f69-433a-be2b-39742276995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.952975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca368e4-9f69-433a-be2b-39742276995a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.953251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca368e4-9f69-433a-be2b-39742276995a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.953568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca368e4-9f69-433a-be2b-39742276995a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:23.954545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca368e4-9f69-433a-be2b-39742276995a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:23.957150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca368e4-9f69-433a-be2b-39742276995a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:23.975088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 990/1999] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:23.979974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:23.992029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7ec18-1a12-45c5-8c45-f759466c2e6b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:23.998559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:23.999215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:23.999515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:23.999803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] PATH_INFO: `/v3/projects/3bc32b9cf64d43faa7612e57a4c12357/users/847cbf239cbb424abc3a7294dd1a05b8/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:24.002629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc32b9cf64d43faa7612e57a4c12357, user_id=847cbf239cbb424abc3a7294dd1a05b8, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:24.005363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:24.016479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1010/2000] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:23 2022] PUT /identity/v3/projects/3bc32b9cf64d43faa7612e57a4c12357/users/847cbf239cbb424abc3a7294dd1a05b8/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:24.029062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7812c23-f98d-471f-91f5-5228777003e1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:24.029314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7812c23-f98d-471f-91f5-5228777003e1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:24.029590 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7812c23-f98d-471f-91f5-5228777003e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:24.059076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-e7812c23-f98d-471f-91f5-5228777003e1 None None] MFA Rules not processed for user `847cbf239cbb424abc3a7294dd1a05b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:24.060307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e7812c23-f98d-471f-91f5-5228777003e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:24.135714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 991/2001] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:24.147499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:24.149363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:24.215310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b378947-713a-4aa1-bbd1-b171cce725a2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:24.220484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bff4a-1671-4926-9949-fa1fdcad4da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:24.221244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bff4a-1671-4926-9949-fa1fdcad4da7 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:24.221353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bff4a-1671-4926-9949-fa1fdcad4da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:24.221617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bff4a-1671-4926-9949-fa1fdcad4da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:24.230752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bff4a-1671-4926-9949-fa1fdcad4da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:24.232151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bff4a-1671-4926-9949-fa1fdcad4da7 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:24.242567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1011/2002] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:24.863960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7812c23-f98d-471f-91f5-5228777003e1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:24.911782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:24.935335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7812c23-f98d-471f-91f5-5228777003e1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:24.941662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:24.942300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:24.942589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:24.942847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] PATH_INFO: `/v3/users/9df8e6679cb84d21bc50ad69f646990f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:24.944088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df8e6679cb84d21bc50ad69f646990f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:24.946087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:25.003984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df8e6679cb84d21bc50ad69f646990f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.011184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df8e6679cb84d21bc50ad69f646990f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.011883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df8e6679cb84d21bc50ad69f646990f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.020803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df8e6679cb84d21bc50ad69f646990f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.032784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df8e6679cb84d21bc50ad69f646990f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.039213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 992/2003] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:24 2022] DELETE /identity/v3/users/9df8e6679cb84d21bc50ad69f646990f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:25.043724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:25.044240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:25.044240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:25.063646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:25.065054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:25.088703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:25.131634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1012/2004] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:25.141921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:25.143918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:25.210513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a10ac0-23af-4177-a481-8583665d1fa2 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:25.214425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:25.215025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:25.215292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:25.215539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:25.224800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:25.226237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:25.235999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 993/2005] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:25.525024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:25.526656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:25.539777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343743e3-d7df-4809-a348-948fca2ef325 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:25.545136 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:25.545793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:25.546071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:25.546342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] PATH_INFO: `/v3/projects/b6d9faed9db344c4a7ef3d87273a4c07` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:25.547492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d9faed9db344c4a7ef3d87273a4c07)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:25.549490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:25.609061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d9faed9db344c4a7ef3d87273a4c07', 'request_id': 'req-4fe9ede6-a449-4bff-9a63-af933504e838'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.616119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d9faed9db344c4a7ef3d87273a4c07', 'request_id': 'req-4fe9ede6-a449-4bff-9a63-af933504e838'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.621850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d9faed9db344c4a7ef3d87273a4c07', 'request_id': 'req-4fe9ede6-a449-4bff-9a63-af933504e838'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.636406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] The token cache is being invalidate because project b6d9faed9db344c4a7ef3d87273a4c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:25.637509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.638289 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.639678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1013/2006] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:25 2022] DELETE /identity/v3/projects/b6d9faed9db344c4a7ef3d87273a4c07 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:25.644827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:25.654657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc3e57-daa6-463b-93ea-0f99ccfef8bb service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:25.721277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:25.722556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:25.722839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:25.723123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] PATH_INFO: `/v3/users/847cbf239cbb424abc3a7294dd1a05b8` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:25.724345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847cbf239cbb424abc3a7294dd1a05b8)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:25.726310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:25.731555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:25.798473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847cbf239cbb424abc3a7294dd1a05b8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.806816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847cbf239cbb424abc3a7294dd1a05b8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.807749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847cbf239cbb424abc3a7294dd1a05b8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.818207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847cbf239cbb424abc3a7294dd1a05b8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.832175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-4e230203-5b14-4dc8-b6c1-7c2eb7e9b7e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847cbf239cbb424abc3a7294dd1a05b8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:25.840071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 994/2007] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:25 2022] DELETE /identity/v3/users/847cbf239cbb424abc3a7294dd1a05b8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:26.145937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.147458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:26.165397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9ede6-a449-4bff-9a63-af933504e838 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:26.226248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:26.226897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.227177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.227440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] PATH_INFO: `/v3/projects/3bc32b9cf64d43faa7612e57a4c12357` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.228797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc32b9cf64d43faa7612e57a4c12357)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:26.230812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:26.288878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc32b9cf64d43faa7612e57a4c12357', 'request_id': 'req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:26.297271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc32b9cf64d43faa7612e57a4c12357', 'request_id': 'req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:26.300212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:26.310976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc32b9cf64d43faa7612e57a4c12357', 'request_id': 'req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:26.326878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] The token cache is being invalidate because project 3bc32b9cf64d43faa7612e57a4c12357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:26.327159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:26.327760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-66c8b6fc-7d51-4445-a6e5-2a03ae5b9d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:26.329752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1014/2008] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:26 2022] DELETE /identity/v3/projects/3bc32b9cf64d43faa7612e57a4c12357 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:26.334811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246207d-40b9-45e9-8d26-f87401f11106 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.335340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246207d-40b9-45e9-8d26-f87401f11106 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.335340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246207d-40b9-45e9-8d26-f87401f11106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.356052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4246207d-40b9-45e9-8d26-f87401f11106 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:26.357396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4246207d-40b9-45e9-8d26-f87401f11106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:26.418282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 995/2009] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:26.426439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b4c04-4269-4c41-96a8-71815576aad6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.426722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b4c04-4269-4c41-96a8-71815576aad6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.427024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b4c04-4269-4c41-96a8-71815576aad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.448524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b35b4c04-4269-4c41-96a8-71815576aad6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:26.449911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b35b4c04-4269-4c41-96a8-71815576aad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:26.459690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1015/2010] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:26.464866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246207d-40b9-45e9-8d26-f87401f11106 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.477781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246207d-40b9-45e9-8d26-f87401f11106 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:26.482679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:26.483273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.483543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.483809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.484861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:26.485156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:26.487221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:26.499117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 996/2011] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:26.504380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b4c04-4269-4c41-96a8-71815576aad6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.514804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b4c04-4269-4c41-96a8-71815576aad6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:26.520870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:26.521578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.521855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.522116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.522874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:26.524843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:26.559745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1016/2012] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:26 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:26.561279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.572029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2cb8-c09e-4342-ba71-8ce3d707ba00 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:26.631548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:26.632207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.632488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.632754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.633407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:26.635356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:26.653939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:26.659259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 997/2013] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:26 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:26.665146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.678708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c01c88-2955-453c-b73f-3768a19af481 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:26.683652 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:26.684249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.684582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.684846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.685466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:26.687073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:26.699404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1017/2014] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:26.708117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.719030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e9388-b9e9-4809-8280-ffe5616d880c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:26.723953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb51d54-78ae-4dcf-ad8d-894993ff6f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:26.724598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb51d54-78ae-4dcf-ad8d-894993ff6f62 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.724859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb51d54-78ae-4dcf-ad8d-894993ff6f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.725122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb51d54-78ae-4dcf-ad8d-894993ff6f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.725755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb51d54-78ae-4dcf-ad8d-894993ff6f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:26.727358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb51d54-78ae-4dcf-ad8d-894993ff6f62 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:26.738660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 998/2015] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:26.742955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.753306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae783670-56bc-424c-b7ce-05d12a549fa3 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:26.757965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:26.758681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.758953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.759313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] PATH_INFO: `/v3/projects/f748009873da4610bcaaae9d6b2d650a/users/cbad13e5f8594a169f60c5416b876239/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.778152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f748009873da4610bcaaae9d6b2d650a, user_id=cbad13e5f8594a169f60c5416b876239, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:26.792597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:26.822639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1018/2016] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:26 2022] PUT /identity/v3/projects/f748009873da4610bcaaae9d6b2d650a/users/cbad13e5f8594a169f60c5416b876239/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:26.834840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd65d3-50d1-48b8-9b3e-6d251806e335 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:26.835446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd65d3-50d1-48b8-9b3e-6d251806e335 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:26.835713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd65d3-50d1-48b8-9b3e-6d251806e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:26.889518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-bedd65d3-50d1-48b8-9b3e-6d251806e335 None None] MFA Rules not processed for user `cbad13e5f8594a169f60c5416b876239`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:26.890748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bedd65d3-50d1-48b8-9b3e-6d251806e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:26.979716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 999/2017] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:26.990841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:26.992711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:27.072544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c6074-9899-4800-a942-816542cb0079 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:27.082404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc83205-b9a4-4e77-9789-8d860f1dca3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:27.083052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83205-b9a4-4e77-9789-8d860f1dca3e service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:27.083323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83205-b9a4-4e77-9789-8d860f1dca3e service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:27.083559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83205-b9a4-4e77-9789-8d860f1dca3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:27.093304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83205-b9a4-4e77-9789-8d860f1dca3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:27.094921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83205-b9a4-4e77-9789-8d860f1dca3e service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:27.105130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1019/2018] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:33.553025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b669f-3633-4e9a-91aa-01fb4962c886 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:33.553861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b669f-3633-4e9a-91aa-01fb4962c886 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:33.554386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b669f-3633-4e9a-91aa-01fb4962c886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:33.574427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0a1b669f-3633-4e9a-91aa-01fb4962c886 None None] MFA Rules not processed for user `d17fb4b6648e436ab20481059f81600a`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:33.575611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0a1b669f-3633-4e9a-91aa-01fb4962c886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:33.651803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1000/2019] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:37.100468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=3,keystone:SELECT=57,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:43.244411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bdbbc-ffe7-4f54-81a2-180499fd4690 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:43.245008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bdbbc-ffe7-4f54-81a2-180499fd4690 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:43.245008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bdbbc-ffe7-4f54-81a2-180499fd4690 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:43.246341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1020/2020] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:51:43.276510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:43.277145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:43.277145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:43.300699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] MFA Rules not processed for user `fbb12187c53a40e79fe0227256346aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:43.303534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:43.314854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bdbbc-ffe7-4f54-81a2-180499fd4690 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:43.317004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0a5bdbbc-ffe7-4f54-81a2-180499fd4690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:43.386334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1001/2021] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:51:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:43.397964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:43.399934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:43.412579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717368df-6f62-4621-8cfa-36457aa823c9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:43.415524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bdbbc-ffe7-4f54-81a2-180499fd4690 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:43.417802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:43.418518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:43.418929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:43.419202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:43.420849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:43.421475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:43.421746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:43.422145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] PATH_INFO: `/v3/users/5c7c0993b20843e4804fa96cd9be34d1` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:43.423331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7c0993b20843e4804fa96cd9be34d1)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:43.436609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:43.440184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:43.441859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:43.452098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1002/2022] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:51:43.474829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7c0993b20843e4804fa96cd9be34d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.475490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7c0993b20843e4804fa96cd9be34d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.476105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7c0993b20843e4804fa96cd9be34d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.482996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7c0993b20843e4804fa96cd9be34d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.491627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c510d101-b3ae-4cb1-b883-5dff65081bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7c0993b20843e4804fa96cd9be34d1'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.508717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1021/2023] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:43 2022] DELETE /identity/v3/users/5c7c0993b20843e4804fa96cd9be34d1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:43.695273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:43.707323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49834-9348-4b6f-87f2-dd272584499b service placement] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:43.711024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:43.794132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:43.794786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:43.795064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:43.795334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] PATH_INFO: `/v3/projects/9475bf21c0234ae5bf55a9c2ba967d33` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:43.796528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9475bf21c0234ae5bf55a9c2ba967d33)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:43.798445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:43.855697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9475bf21c0234ae5bf55a9c2ba967d33', 'request_id': 'req-5a514526-f956-45fb-ab17-da11a16dbbb0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.864304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9475bf21c0234ae5bf55a9c2ba967d33', 'request_id': 'req-5a514526-f956-45fb-ab17-da11a16dbbb0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.871468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9475bf21c0234ae5bf55a9c2ba967d33', 'request_id': 'req-5a514526-f956-45fb-ab17-da11a16dbbb0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.884749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] The token cache is being invalidate because project 9475bf21c0234ae5bf55a9c2ba967d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:43.885097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.885713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-5a514526-f956-45fb-ab17-da11a16dbbb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:43.887374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1003/2024] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:43 2022] DELETE /identity/v3/projects/9475bf21c0234ae5bf55a9c2ba967d33 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:43.892198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ec577-e350-4a58-8fb8-af14ecee048e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:43.892514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ec577-e350-4a58-8fb8-af14ecee048e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:43.892791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ec577-e350-4a58-8fb8-af14ecee048e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:43.913237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b65ec577-e350-4a58-8fb8-af14ecee048e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:43.914454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b65ec577-e350-4a58-8fb8-af14ecee048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:43.974265 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1022/2025] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:43.983607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad07d2-d0f8-4a58-99d6-09c72210f50a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:43.983880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad07d2-d0f8-4a58-99d6-09c72210f50a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:43.984146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad07d2-d0f8-4a58-99d6-09c72210f50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.004825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4aad07d2-d0f8-4a58-99d6-09c72210f50a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:44.006035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4aad07d2-d0f8-4a58-99d6-09c72210f50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:44.015214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1004/2026] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:44.019896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ec577-e350-4a58-8fb8-af14ecee048e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.030036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ec577-e350-4a58-8fb8-af14ecee048e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.035447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.036039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.036321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.036592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.037226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:44.037521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.039101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.051252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1023/2027] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:44.055643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad07d2-d0f8-4a58-99d6-09c72210f50a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.066234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad07d2-d0f8-4a58-99d6-09c72210f50a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.071951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.072863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.073141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.073388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.074044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.075970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.106851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1005/2028] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:44 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:44.111555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.123213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87740555-9677-431c-a3f4-42a10822d0fb admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.127082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:44.188077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.188718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.188997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.189270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.189890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.191788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.215226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1024/2029] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:44 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:44.219473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.229864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb814c3-fe26-49c2-9ff8-59ce720b5842 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.234505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.235124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.235395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.235691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.236386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.238015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.248093 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1006/2030] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:44.252109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.262165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f303f-21d2-4128-a1c8-753d148752f5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.266771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d81cb-7bbc-44fc-9c6f-5e3ef63da724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.267358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d81cb-7bbc-44fc-9c6f-5e3ef63da724 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.267627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d81cb-7bbc-44fc-9c6f-5e3ef63da724 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.267888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d81cb-7bbc-44fc-9c6f-5e3ef63da724 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.268564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d81cb-7bbc-44fc-9c6f-5e3ef63da724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.270487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d81cb-7bbc-44fc-9c6f-5e3ef63da724 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.280920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1025/2031] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:44.285194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.295136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff2e8f-3aa8-45cb-aebf-ef5338dabab9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.300469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.301103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.301372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.301634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] PATH_INFO: `/v3/projects/68dfc9db6439443c92d32c6dd0d35662/users/af5289a3bf994af3986eb8470b84ede7/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.319172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68dfc9db6439443c92d32c6dd0d35662, user_id=af5289a3bf994af3986eb8470b84ede7, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.322521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.338756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1007/2032] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:44 2022] PUT /identity/v3/projects/68dfc9db6439443c92d32c6dd0d35662/users/af5289a3bf994af3986eb8470b84ede7/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:44.349611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bc7eb-21ea-4783-afbd-e4aaa292608f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.349924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bc7eb-21ea-4783-afbd-e4aaa292608f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.350141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bc7eb-21ea-4783-afbd-e4aaa292608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.379456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-253bc7eb-21ea-4783-afbd-e4aaa292608f None None] MFA Rules not processed for user `af5289a3bf994af3986eb8470b84ede7`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:44.382713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-253bc7eb-21ea-4783-afbd-e4aaa292608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:44.476303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1026/2033] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:44 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:44.480180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.523085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:44.552845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d0b17-736d-420e-af73-893facbaed29 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.557984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.558597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.559507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.559784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.560447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.562398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.594571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1008/2034] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:44 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:44.595954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bc7eb-21ea-4783-afbd-e4aaa292608f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.607317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bc7eb-21ea-4783-afbd-e4aaa292608f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.667743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.668396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.668672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.668937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.669557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.671491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.694405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1027/2035] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:44 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:44.698848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.709500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c2039-84d0-4d80-8013-0ded2c6a18f1 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.714666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.715265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.715595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.715866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.716535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.718222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.729016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1009/2036] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:44.733605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.740546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:44.745087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16685be1-fde3-4ec8-9b5c-39713f175da2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.750613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.751276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.751552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.751819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] PATH_INFO: `/v3/projects/f591c1b7bf3d44989728996f9628f9a5/users/97d268310b234834b24aa34dee82d8bf/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.766811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f591c1b7bf3d44989728996f9628f9a5, user_id=97d268310b234834b24aa34dee82d8bf, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.769804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.784124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1028/2037] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:44 2022] PUT /identity/v3/projects/f591c1b7bf3d44989728996f9628f9a5/users/97d268310b234834b24aa34dee82d8bf/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:44.788430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.799416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115a79f-b18e-42e3-b4ae-332d8da712a3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.860239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a80ff-744d-48fe-b521-bfe75ab605c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.860887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a80ff-744d-48fe-b521-bfe75ab605c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.861164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a80ff-744d-48fe-b521-bfe75ab605c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.861424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a80ff-744d-48fe-b521-bfe75ab605c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.862067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a80ff-744d-48fe-b521-bfe75ab605c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.863685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a80ff-744d-48fe-b521-bfe75ab605c1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.874109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1010/2038] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:44.880442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:44.888779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacec2c-aba1-4bf9-bc2b-3ed6b4485dbc admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:44.893517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:44.894501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.894777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.895047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] PATH_INFO: `/v3/projects/f591c1b7bf3d44989728996f9628f9a5/users/97d268310b234834b24aa34dee82d8bf/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.897078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f591c1b7bf3d44989728996f9628f9a5, user_id=97d268310b234834b24aa34dee82d8bf, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:44.899712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:44.909908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1029/2039] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:44 2022] PUT /identity/v3/projects/f591c1b7bf3d44989728996f9628f9a5/users/97d268310b234834b24aa34dee82d8bf/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:44.921419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787921-9927-4c2d-bbc2-7dd0bf542c4a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:44.921711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787921-9927-4c2d-bbc2-7dd0bf542c4a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:44.921984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787921-9927-4c2d-bbc2-7dd0bf542c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:44.950094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3c787921-9927-4c2d-bbc2-7dd0bf542c4a None None] MFA Rules not processed for user `97d268310b234834b24aa34dee82d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:44.951338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3c787921-9927-4c2d-bbc2-7dd0bf542c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:45.024706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1011/2040] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:44 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:45.035698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:45.037482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:45.102944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cbd26-cf79-4790-abfe-19036ada0e7a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:45.108352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d3a71-ebbe-4397-8433-2cde7a3b0033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:45.109000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d3a71-ebbe-4397-8433-2cde7a3b0033 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:45.109304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d3a71-ebbe-4397-8433-2cde7a3b0033 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:45.109565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d3a71-ebbe-4397-8433-2cde7a3b0033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:45.119230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d3a71-ebbe-4397-8433-2cde7a3b0033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:45.120809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d3a71-ebbe-4397-8433-2cde7a3b0033 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:45.130683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1030/2041] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:45.320555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787921-9927-4c2d-bbc2-7dd0bf542c4a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:45.359125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:45.396901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787921-9927-4c2d-bbc2-7dd0bf542c4a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:45.404133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:45.404868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:45.405139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:45.405418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] PATH_INFO: `/v3/users/af5289a3bf994af3986eb8470b84ede7` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:45.406657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5289a3bf994af3986eb8470b84ede7)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:45.408645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:45.470557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5289a3bf994af3986eb8470b84ede7'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:45.482655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5289a3bf994af3986eb8470b84ede7'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:45.483451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5289a3bf994af3986eb8470b84ede7'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:45.493474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5289a3bf994af3986eb8470b84ede7'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:45.510076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5289a3bf994af3986eb8470b84ede7'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:45.518566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1012/2042] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:45 2022] DELETE /identity/v3/users/af5289a3bf994af3986eb8470b84ede7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:45.523394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:45.524032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:45.524032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:45.545262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:45.546695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:45.658760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1031/2043] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:45.666111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:45.669162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:45.750199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f25a40-f157-470e-8a4a-a15a7ed85d7a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:45.753824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:45.754429 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:45.754698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:45.754963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:45.764915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:45.766361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:45.776993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1013/2044] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:46.251617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:46.254203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:46.280619 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24bb0b-8b0e-41c3-bf03-739e280ce07d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:46.289536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:46.290178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:46.290449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:46.290712 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] PATH_INFO: `/v3/projects/68dfc9db6439443c92d32c6dd0d35662` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:46.292020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68dfc9db6439443c92d32c6dd0d35662)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:46.293958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:46.354444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68dfc9db6439443c92d32c6dd0d35662', 'request_id': 'req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.360832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68dfc9db6439443c92d32c6dd0d35662', 'request_id': 'req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.367248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68dfc9db6439443c92d32c6dd0d35662', 'request_id': 'req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.371257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:46.388306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] The token cache is being invalidate because project 68dfc9db6439443c92d32c6dd0d35662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:46.388506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.389859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.392375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1032/2045] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:46 2022] DELETE /identity/v3/projects/68dfc9db6439443c92d32c6dd0d35662 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:46.397899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:46.415266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e17782-7ab7-4322-b4ce-bf348fca68c9 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:46.467762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:46.478909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:46.479662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:46.479947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:46.480216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] PATH_INFO: `/v3/users/97d268310b234834b24aa34dee82d8bf` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:46.481423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d268310b234834b24aa34dee82d8bf)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:46.483370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:46.536221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d268310b234834b24aa34dee82d8bf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.548328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d268310b234834b24aa34dee82d8bf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.549297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d268310b234834b24aa34dee82d8bf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.559164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d268310b234834b24aa34dee82d8bf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.571928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-09c1bd1a-44c4-45d6-a819-155b6e3fac63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d268310b234834b24aa34dee82d8bf'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:46.578160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1014/2046] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:46 2022] DELETE /identity/v3/users/97d268310b234834b24aa34dee82d8bf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:47.071766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.073963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:47.087556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283e4a7-faf5-4ddb-be13-7f58dd4a6e83 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.146997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.147844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.148129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.148428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] PATH_INFO: `/v3/projects/f591c1b7bf3d44989728996f9628f9a5` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.149567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f591c1b7bf3d44989728996f9628f9a5)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.151628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.199863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f591c1b7bf3d44989728996f9628f9a5', 'request_id': 'req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:47.206517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f591c1b7bf3d44989728996f9628f9a5', 'request_id': 'req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:47.212543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f591c1b7bf3d44989728996f9628f9a5', 'request_id': 'req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:47.225395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] The token cache is being invalidate because project f591c1b7bf3d44989728996f9628f9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:47.225676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:47.226279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-72de1e3a-7ef2-4fe8-b05b-9fd91c7fe38d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:47.227919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1033/2047] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:47 2022] DELETE /identity/v3/projects/f591c1b7bf3d44989728996f9628f9a5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:47.233738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa519b-1fc0-44a8-9f64-cad49af679ca None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.234024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa519b-1fc0-44a8-9f64-cad49af679ca None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.234317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa519b-1fc0-44a8-9f64-cad49af679ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.254541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3aaa519b-1fc0-44a8-9f64-cad49af679ca None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:47.255753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3aaa519b-1fc0-44a8-9f64-cad49af679ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:47.319390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1015/2048] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:47.327309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99118df9-c2f4-4668-888a-00ec1d80a36f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.327562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99118df9-c2f4-4668-888a-00ec1d80a36f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.327866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99118df9-c2f4-4668-888a-00ec1d80a36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.348002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-99118df9-c2f4-4668-888a-00ec1d80a36f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:47.349251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-99118df9-c2f4-4668-888a-00ec1d80a36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:47.357980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1034/2049] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:47.362186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa519b-1fc0-44a8-9f64-cad49af679ca None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.372126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa519b-1fc0-44a8-9f64-cad49af679ca None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.377283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.377878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.378149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.378408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.379054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:47.379346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.380975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.393365 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1016/2050] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:47.398179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99118df9-c2f4-4668-888a-00ec1d80a36f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.407290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99118df9-c2f4-4668-888a-00ec1d80a36f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.411900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.412524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.412801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.413069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.413699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.415601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.449568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1035/2051] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:47 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:47.450751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.461645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8cb5-a306-4e61-aea3-647e84cb2a9d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.528014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:47.535739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.536510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.536888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.537405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.538303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.541195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.565976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1017/2052] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:47 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:47.570172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.581028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39899e-93fa-4f3f-a5fb-8b19aa0b6dc1 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.586322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.587415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.587694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.588132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.589116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.590719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.601644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1036/2053] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:47.605739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.616297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2a5f7-9f72-40be-aaf2-bb772d4cb28e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.621124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28698598-5833-43ac-b07e-3199703cb41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.621730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28698598-5833-43ac-b07e-3199703cb41e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.622008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28698598-5833-43ac-b07e-3199703cb41e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.622280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28698598-5833-43ac-b07e-3199703cb41e admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.622926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28698598-5833-43ac-b07e-3199703cb41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.624575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28698598-5833-43ac-b07e-3199703cb41e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.635421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1018/2054] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:47.641668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.652068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d37e0c-f3e5-4c91-8c4c-5a5dbaa7e217 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.656881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.657521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.657789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.658050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] PATH_INFO: `/v3/projects/e86102d719544d2eb72e270a14cfff28/users/cef82862bd454eda9b3a3310360d6296/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.672747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86102d719544d2eb72e270a14cfff28, user_id=cef82862bd454eda9b3a3310360d6296, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.675436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.690450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1037/2055] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:47 2022] PUT /identity/v3/projects/e86102d719544d2eb72e270a14cfff28/users/cef82862bd454eda9b3a3310360d6296/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:47.701030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d8b3-4876-43f7-9381-b4d344220a11 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.701310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d8b3-4876-43f7-9381-b4d344220a11 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.701578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d8b3-4876-43f7-9381-b4d344220a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.730406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-94e5d8b3-4876-43f7-9381-b4d344220a11 None None] MFA Rules not processed for user `cef82862bd454eda9b3a3310360d6296`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:47.731628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-94e5d8b3-4876-43f7-9381-b4d344220a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:47.797126 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1019/2056] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:47.801904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.822386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:47.867934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70431cd9-28d3-4ccc-b2df-986174361be1 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.874321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.874897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.875194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.875453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.876068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:47.878361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:47.907608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1038/2057] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:47 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:47.911825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5d8b3-4876-43f7-9381-b4d344220a11 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:47.923202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5d8b3-4876-43f7-9381-b4d344220a11 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:47.996428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:47.997419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:47.997889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:47.998329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:47.999145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:48.001305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:48.034955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1020/2058] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:47 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:48.038793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:48.057038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc5d17-b10b-4f59-a182-a2db0cdc1722 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:48.075519 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:48.076164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:48.076434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:48.076698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:48.077334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:48.079122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:48.090990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1039/2059] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:48.096673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:48.108870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc62ed7-c1af-4f92-bfe2-06687f900c27 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:48.116703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:48.117484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:48.117807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:48.118083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] PATH_INFO: `/v3/projects/ec78966847d544c08910c5984512acb6/users/2c19696f85dd4038b642675268da9257/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:48.135800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec78966847d544c08910c5984512acb6, user_id=2c19696f85dd4038b642675268da9257, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:48.138644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:48.156153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1021/2060] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:48 2022] PUT /identity/v3/projects/ec78966847d544c08910c5984512acb6/users/2c19696f85dd4038b642675268da9257/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:48.163095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:48.174001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280016bf-bb48-4b41-90b9-2677d281f2d5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:48.232421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6141043-515d-4031-81e6-c6906afaf954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:48.233054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6141043-515d-4031-81e6-c6906afaf954 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:48.233335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6141043-515d-4031-81e6-c6906afaf954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:48.233618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6141043-515d-4031-81e6-c6906afaf954 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:48.234254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6141043-515d-4031-81e6-c6906afaf954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:48.235858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6141043-515d-4031-81e6-c6906afaf954 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:48.246138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1040/2061] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:48.250998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:48.262446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121febc-7959-4e16-9f67-70eb6eda3371 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:48.267897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:48.268611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:48.269217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:48.269482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] PATH_INFO: `/v3/projects/ec78966847d544c08910c5984512acb6/users/2c19696f85dd4038b642675268da9257/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:48.271431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec78966847d544c08910c5984512acb6, user_id=2c19696f85dd4038b642675268da9257, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:48.274208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:48.284569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1022/2062] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:48 2022] PUT /identity/v3/projects/ec78966847d544c08910c5984512acb6/users/2c19696f85dd4038b642675268da9257/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:48.296871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51170-d94f-45a1-ac15-9e4c8a23d3a3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:48.297159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51170-d94f-45a1-ac15-9e4c8a23d3a3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:48.297431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51170-d94f-45a1-ac15-9e4c8a23d3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:48.325914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-efc51170-d94f-45a1-ac15-9e4c8a23d3a3 None None] MFA Rules not processed for user `2c19696f85dd4038b642675268da9257`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:48.327189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-efc51170-d94f-45a1-ac15-9e4c8a23d3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:48.410806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1041/2063] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:48.423513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:48.425804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:48.451211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:48.490909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748a8a-985f-406c-973b-5c02843e6c4a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:48.494915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dba9d8-4cf1-4192-81e7-054c777caca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:48.495532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dba9d8-4cf1-4192-81e7-054c777caca3 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:48.495804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dba9d8-4cf1-4192-81e7-054c777caca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:48.496074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dba9d8-4cf1-4192-81e7-054c777caca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:48.507956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dba9d8-4cf1-4192-81e7-054c777caca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:48.509469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dba9d8-4cf1-4192-81e7-054c777caca3 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:48.520406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1023/2064] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:48.994948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51170-d94f-45a1-ac15-9e4c8a23d3a3 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:49.012361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:49.064904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51170-d94f-45a1-ac15-9e4c8a23d3a3 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:49.069595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:49.070241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:49.070516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:49.070786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] PATH_INFO: `/v3/users/cef82862bd454eda9b3a3310360d6296` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:49.071941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef82862bd454eda9b3a3310360d6296)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:49.074270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:49.148063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef82862bd454eda9b3a3310360d6296'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.148801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef82862bd454eda9b3a3310360d6296'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.149353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef82862bd454eda9b3a3310360d6296'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.160988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef82862bd454eda9b3a3310360d6296'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.181447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef82862bd454eda9b3a3310360d6296'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.200897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1042/2065] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:48 2022] DELETE /identity/v3/users/cef82862bd454eda9b3a3310360d6296 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:49.205421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:49.205702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:49.205975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:49.229632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:49.230910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:49.311354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1024/2066] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:49.325268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:49.327603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:49.402915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:49.404900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:49.423113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d10bd-1784-45d6-8279-9375cbe49137 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:49.425341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b250d5-4861-4f66-b325-f03618c13486 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:49.428452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:49.429095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:49.429354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59141831-4e5a-4cbf-8725-e003661aaf8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:49.429471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:49.429629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] PATH_INFO: `/v3/users/cbad13e5f8594a169f60c5416b876239` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:49.430108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141831-4e5a-4cbf-8725-e003661aaf8f service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:49.430455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141831-4e5a-4cbf-8725-e003661aaf8f service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:49.430903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141831-4e5a-4cbf-8725-e003661aaf8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:49.431008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbad13e5f8594a169f60c5416b876239)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:49.433178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:49.447335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59141831-4e5a-4cbf-8725-e003661aaf8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:49.449161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59141831-4e5a-4cbf-8725-e003661aaf8f service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:49.476639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1043/2067] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:49.515863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbad13e5f8594a169f60c5416b876239'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.524678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbad13e5f8594a169f60c5416b876239'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.525471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbad13e5f8594a169f60c5416b876239'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.538503 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbad13e5f8594a169f60c5416b876239'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.571115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbad13e5f8594a169f60c5416b876239'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.584488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1025/2068] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:49 2022] DELETE /identity/v3/users/cbad13e5f8594a169f60c5416b876239 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:49.586531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:49.586722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:49.586992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:49.611612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:49.613758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:49.665332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:49.699544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1044/2069] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:49.709428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:49.710913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:49.757747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:49.783219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ba84b-8e8a-4ee3-904d-106855b66bcd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:49.787160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:49.787942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:49.788234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:49.788566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:49.798088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:49.799531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:49.810455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1026/2070] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:49.892182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:49.894197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:49.910125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddab5d-8e02-4dec-a95e-1ed4906aca2b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:49.915244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:49.915898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:49.916174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:49.916789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] PATH_INFO: `/v3/projects/e86102d719544d2eb72e270a14cfff28` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:49.918559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86102d719544d2eb72e270a14cfff28)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:49.921328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:49.986753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86102d719544d2eb72e270a14cfff28', 'request_id': 'req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.993217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86102d719544d2eb72e270a14cfff28', 'request_id': 'req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:49.999162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86102d719544d2eb72e270a14cfff28', 'request_id': 'req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.015830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] The token cache is being invalidate because project e86102d719544d2eb72e270a14cfff28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:50.016122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.016980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.019090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1045/2071] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:49 2022] DELETE /identity/v3/projects/e86102d719544d2eb72e270a14cfff28 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:50.023263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.037023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ba085-c38c-4f7a-98bf-b28f4938e75e service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.146419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.147301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.147796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.148239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] PATH_INFO: `/v3/users/2c19696f85dd4038b642675268da9257` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.149868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c19696f85dd4038b642675268da9257)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:50.155196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:50.240489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c19696f85dd4038b642675268da9257'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.240939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.240939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:50.248738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c19696f85dd4038b642675268da9257'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.251630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c19696f85dd4038b642675268da9257'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.266366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c19696f85dd4038b642675268da9257'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.267952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc9d8c-7ad2-4de1-8e32-4deff00052b5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.281969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.282930 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.285903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.286284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] PATH_INFO: `/v3/projects/f748009873da4610bcaaae9d6b2d650a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.289541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f748009873da4610bcaaae9d6b2d650a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:50.290120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-29263e41-fbb8-43b1-9495-3314bd4b4ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c19696f85dd4038b642675268da9257'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.293392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:50.304609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1027/2072] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:50 2022] DELETE /identity/v3/users/2c19696f85dd4038b642675268da9257 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:50.439438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f748009873da4610bcaaae9d6b2d650a', 'request_id': 'req-43a81070-084a-4c79-999c-dfcefb338d69'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.452025 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f748009873da4610bcaaae9d6b2d650a', 'request_id': 'req-43a81070-084a-4c79-999c-dfcefb338d69'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.459870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f748009873da4610bcaaae9d6b2d650a', 'request_id': 'req-43a81070-084a-4c79-999c-dfcefb338d69'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.494662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] The token cache is being invalidate because project f748009873da4610bcaaae9d6b2d650a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:50.495245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.496096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-43a81070-084a-4c79-999c-dfcefb338d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.502420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1046/2073] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:50 2022] DELETE /identity/v3/projects/f748009873da4610bcaaae9d6b2d650a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:50.504288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.504633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.504917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.568721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:50.570284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:50.638134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:50.658880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1028/2074] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:50.660701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85a34c-673b-4adf-9619-8a3e8e888515 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.661091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85a34c-673b-4adf-9619-8a3e8e888515 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.661464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85a34c-673b-4adf-9619-8a3e8e888515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.689777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-1a85a34c-673b-4adf-9619-8a3e8e888515 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:50.690977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1a85a34c-673b-4adf-9619-8a3e8e888515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:50.701613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.704585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1047/2075] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:50.705874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:50.713218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85a34c-673b-4adf-9619-8a3e8e888515 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.729954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85a34c-673b-4adf-9619-8a3e8e888515 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.733549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77978-b125-4814-a17b-ea184fd8ee2f None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.735116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.735856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.736178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.736547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.737380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:50.737734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:50.739524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.739524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.739759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:50.739849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.739849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] PATH_INFO: `/v3/projects/ec78966847d544c08910c5984512acb6` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.740960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec78966847d544c08910c5984512acb6)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:50.742854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:50.756311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1048/2076] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:50.758942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.770572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e557c-f23b-41e4-8a91-64ce5911c5ed admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.778599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.779206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.779525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.779917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.780666 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:50.782601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:50.805273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec78966847d544c08910c5984512acb6', 'request_id': 'req-c9a95855-31cd-4852-988b-535c01032be5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.811891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec78966847d544c08910c5984512acb6', 'request_id': 'req-c9a95855-31cd-4852-988b-535c01032be5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.819445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1049/2077] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:50 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:50.819774 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec78966847d544c08910c5984512acb6', 'request_id': 'req-c9a95855-31cd-4852-988b-535c01032be5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.821070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.825662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:50.837670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] The token cache is being invalidate because project ec78966847d544c08910c5984512acb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:50.837940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.839328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c9a95855-31cd-4852-988b-535c01032be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:50.841477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1029/2078] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:50 2022] DELETE /identity/v3/projects/ec78966847d544c08910c5984512acb6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:50.846189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316bcea-3c5b-405c-9fed-6a471c2169a8 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.846476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316bcea-3c5b-405c-9fed-6a471c2169a8 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.846738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316bcea-3c5b-405c-9fed-6a471c2169a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.872109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3316bcea-3c5b-405c-9fed-6a471c2169a8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:50.873425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3316bcea-3c5b-405c-9fed-6a471c2169a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:50.892361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090d517-98af-4d19-bf6e-d3318939e57b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.897694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.898355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.898638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.898969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.899596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:50.901611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:50.929714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1050/2079] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:50 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:50.933894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.935483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:50.938895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1030/2080] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:50.954587 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace384a-9d14-4076-aec9-e99804a3751a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.955884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace384a-9d14-4076-aec9-e99804a3751a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.956354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace384a-9d14-4076-aec9-e99804a3751a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.959221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6271d-2aa4-4389-a1f5-c7ad1f46b2d0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.965164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.966029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.966299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.966565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:50.967241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:50.968917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:50.980212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1051/2081] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:50.983554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4ace384a-9d14-4076-aec9-e99804a3751a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:50.984097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.984806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4ace384a-9d14-4076-aec9-e99804a3751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:50.993852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958ec2-365a-4f5f-ae01-11ee80ed32db admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:50.993995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1031/2082] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:50.998074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace384a-9d14-4076-aec9-e99804a3751a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:50.998731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:50.999389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:50.999666 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:50.999935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.000604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.002181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.006908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace384a-9d14-4076-aec9-e99804a3751a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.012464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.012805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1052/2083] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:51.013023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.013318 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.013709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.014216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:51.014508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.016172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.016999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.027145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3173ab-5a6b-43a4-8c25-e24c88d026bc admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.028619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1032/2084] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:51.032217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac24f24-6a84-4229-bb17-bed57aba9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.032697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.032879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24f24-6a84-4229-bb17-bed57aba9fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.033148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24f24-6a84-4229-bb17-bed57aba9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.033487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac24f24-6a84-4229-bb17-bed57aba9fbc admin admin] PATH_INFO: `/v3/projects/37fcea6a1e024829bf68678aa5b3a263/users/29aea39f1249420c8585c1685d993dcd/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.042287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb61b0d-71a1-40f3-8d07-9bc433ec968a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.046808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.047383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.047654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.047915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.048572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.048793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac24f24-6a84-4229-bb17-bed57aba9fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fcea6a1e024829bf68678aa5b3a263, user_id=29aea39f1249420c8585c1685d993dcd, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.050441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.051554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac24f24-6a84-4229-bb17-bed57aba9fbc admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.066046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1053/2085] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:51 2022] PUT /identity/v3/projects/37fcea6a1e024829bf68678aa5b3a263/users/29aea39f1249420c8585c1685d993dcd/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:51.076890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e0e9-b957-4ee8-a23c-08b67da2b480 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.077449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e0e9-b957-4ee8-a23c-08b67da2b480 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.077894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e0e9-b957-4ee8-a23c-08b67da2b480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.087189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1033/2086] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:51.091421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.111485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-03c9e0e9-b957-4ee8-a23c-08b67da2b480 None None] MFA Rules not processed for user `29aea39f1249420c8585c1685d993dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:51.113013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-03c9e0e9-b957-4ee8-a23c-08b67da2b480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:51.140344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:51.187913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2015be-1cd0-4a5b-85fe-126d94d12f80 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.195903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.196648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.196889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1054/2087] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:51.201033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.201402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.202217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.202397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9e0e9-b957-4ee8-a23c-08b67da2b480 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.204498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.218344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9e0e9-b957-4ee8-a23c-08b67da2b480 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.225765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.226940 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.227279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.227627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.228869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.231219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.235661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1034/2088] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:51.254209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.268313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1055/2089] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:51.269114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1086706-5a63-4109-a711-d3c07a7245af admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.270756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.286474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:51.330241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.330857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.331135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.331406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.332131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.333869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.350409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1035/2090] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:51.352776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1fcba-9106-4a0a-8b62-73e02a65d08f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.354557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.358919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.359664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.360329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.360642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.362018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.364033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.366154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46ccd-56d9-4bca-a241-8c68aaa8c6ad admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.372029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.372752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.373097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.374051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.374987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.377940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.392508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1036/2091] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:51.400311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1056/2092] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:51.400589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.405336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.411326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d7ca5-3787-448f-9282-c47e3864e468 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.416658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37874d-e073-47bb-a94e-e7b0edb96890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.417439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37874d-e073-47bb-a94e-e7b0edb96890 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.417585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbae9d9-8c1b-42f7-bd39-c8e019fd3306 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.417763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37874d-e073-47bb-a94e-e7b0edb96890 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.417995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37874d-e073-47bb-a94e-e7b0edb96890 admin admin] PATH_INFO: `/v3/projects/0cd4591fd4944500b0697cb9abed7247/users/7f41e62cd2974352a828df4dd03f3613/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.422679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.423278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.424010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.424305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.424945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.426542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.434203 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37874d-e073-47bb-a94e-e7b0edb96890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd4591fd4944500b0697cb9abed7247, user_id=7f41e62cd2974352a828df4dd03f3613, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.437280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37874d-e073-47bb-a94e-e7b0edb96890 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.438378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1057/2093] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:51.443030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.453695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1037/2094] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:51 2022] PUT /identity/v3/projects/0cd4591fd4944500b0697cb9abed7247/users/7f41e62cd2974352a828df4dd03f3613/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:51.455538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc565f-8f4c-4da9-9fc7-b66a1d525cd8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.464885 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067f6f5-a3a4-433c-855f-4d8049e7e5db None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.465135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067f6f5-a3a4-433c-855f-4d8049e7e5db None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.465392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067f6f5-a3a4-433c-855f-4d8049e7e5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.496975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1067f6f5-a3a4-433c-855f-4d8049e7e5db None None] MFA Rules not processed for user `7f41e62cd2974352a828df4dd03f3613`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:51.498198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1067f6f5-a3a4-433c-855f-4d8049e7e5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:51.525332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.526305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.526749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.527496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.528252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.530087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.544959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1058/2095] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:51.549282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.573892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1038/2096] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:51.578492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067f6f5-a3a4-433c-855f-4d8049e7e5db None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.586451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765cc86-75b8-42b7-8db9-e7b88304c8d6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.590294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067f6f5-a3a4-433c-855f-4d8049e7e5db None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.596886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.597504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.597780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.598049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.600426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.600564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc840ed-f757-41a6-b7e1-8ec9b2f801fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.600564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc840ed-f757-41a6-b7e1-8ec9b2f801fb admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.600564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc840ed-f757-41a6-b7e1-8ec9b2f801fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.600884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc840ed-f757-41a6-b7e1-8ec9b2f801fb admin admin] PATH_INFO: `/v3/projects/6a9a05818ab64b8fb6de685453b2dad8/users/4edf938227b94bc1982f581e52d6c8b5/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.601357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.621251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:51.628400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc840ed-f757-41a6-b7e1-8ec9b2f801fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9a05818ab64b8fb6de685453b2dad8, user_id=4edf938227b94bc1982f581e52d6c8b5, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.632824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc840ed-f757-41a6-b7e1-8ec9b2f801fb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.655804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1039/2097] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:51.656907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.660308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1059/2098] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:51 2022] PUT /identity/v3/projects/6a9a05818ab64b8fb6de685453b2dad8/users/4edf938227b94bc1982f581e52d6c8b5/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:51.739517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.739911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.740462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.757704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949bea3-97d9-40c3-b6c9-f18ed85154cf admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.804783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] MFA Rules not processed for user `4edf938227b94bc1982f581e52d6c8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:51.806372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:51.877427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:51.890624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.891320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.891722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.892412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.892636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.893017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1060/2099] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:51.894475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.894939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.897266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:51.912334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9c486-ffdc-4b6a-8408-c9abd5d3ccab None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.917503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.918215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.918548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.918876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] PATH_INFO: `/v3/users/c9e4d103e1af4d07af4ea5c2fb1e51f3` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.919454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1040/2100] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:51.920052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e4d103e1af4d07af4ea5c2fb1e51f3)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.920309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:51.922029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.931656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48a965-c24c-4468-90c4-8fbe71c15dae admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:51.936749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:51.937337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:51.937608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:51.937873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:51.938489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:51.940608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:51.983556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1041/2101] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:51 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:51.983556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.002455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e4d103e1af4d07af4ea5c2fb1e51f3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.003167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e4d103e1af4d07af4ea5c2fb1e51f3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.004194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e4d103e1af4d07af4ea5c2fb1e51f3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.013079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e4d103e1af4d07af4ea5c2fb1e51f3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.029589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e4d103e1af4d07af4ea5c2fb1e51f3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.048040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1061/2102] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:51 2022] DELETE /identity/v3/users/c9e4d103e1af4d07af4ea5c2fb1e51f3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:52.048975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.049955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8204c08-bcaf-4ec9-b800-683f7b11b226 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.050392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.063175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:52.110311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26930706-9a81-4038-b174-d488a801ca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.111023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930706-9a81-4038-b174-d488a801ca81 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.111366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930706-9a81-4038-b174-d488a801ca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.111717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930706-9a81-4038-b174-d488a801ca81 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.112471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26930706-9a81-4038-b174-d488a801ca81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.114167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26930706-9a81-4038-b174-d488a801ca81 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.121362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e882f7-295b-4ee4-b9db-abd69e0040c4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.125310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1042/2103] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:52.126463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.127110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.127334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e7e57-0455-4c48-9090-7c4f239bc490 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.127446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.127737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.127840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e7e57-0455-4c48-9090-7c4f239bc490 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.128070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e7e57-0455-4c48-9090-7c4f239bc490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.128549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.131074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.153516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f86e7e57-0455-4c48-9090-7c4f239bc490 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:52.154820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f86e7e57-0455-4c48-9090-7c4f239bc490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.161888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1062/2104] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:52 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:52.161888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.161888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.162962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1043/2105] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:52.168532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e7e57-0455-4c48-9090-7c4f239bc490 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.174820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e7e57-0455-4c48-9090-7c4f239bc490 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.179557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.180150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.180480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.180755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.180918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f613a-ff21-4a3e-8724-acc032409143 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.181392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.183468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.187392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.188137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.188585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.188977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] PATH_INFO: `/v3/projects/9bd1f0282cf840daacc54b94b0fda178/users/4664852d23ca4d07b67bc208e8237b9f/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.194602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1044/2106] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:52.195412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.197059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.205691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1f0282cf840daacc54b94b0fda178, user_id=4664852d23ca4d07b67bc208e8237b9f, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.208336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.221401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1063/2107] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:52 2022] PUT /identity/v3/projects/9bd1f0282cf840daacc54b94b0fda178/users/4664852d23ca4d07b67bc208e8237b9f/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:52.222214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.255513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa93fe-28d0-4963-9add-8344cbc31278 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.258806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:52.290316 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cab5-e8bc-46c5-ab58-b268070bbe83 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.310307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.310599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.311134 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.311273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.311471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.311583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.311729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.312003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] PATH_INFO: `/v3/projects/04812c1d0be74da299712dc3d91f6249/users/4cd6c058741c4f1f906aa441e72cbe5b/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.323527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.325147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.339879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04812c1d0be74da299712dc3d91f6249, user_id=4cd6c058741c4f1f906aa441e72cbe5b, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.342952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.344911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1045/2108] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:52.345637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.360304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a5735-744d-41b0-824d-e4ad958df87f service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.365655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.366362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.366765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.367099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.367857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.369659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.378973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1064/2109] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:52 2022] PUT /identity/v3/projects/04812c1d0be74da299712dc3d91f6249/users/4cd6c058741c4f1f906aa441e72cbe5b/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:52.383781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.384198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1046/2110] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:52.387482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.405036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e89ea6-27a2-429f-baba-48ec3bbfe4d9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.474036 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:52.492016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadb11f-d684-4e5a-a33b-ff87259f7bb5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.495791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f4106-2e8c-4a97-8b4e-3cf6cbd37907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.496688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f4106-2e8c-4a97-8b4e-3cf6cbd37907 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.496978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f4106-2e8c-4a97-8b4e-3cf6cbd37907 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.497341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f4106-2e8c-4a97-8b4e-3cf6cbd37907 admin admin] PATH_INFO: `/v3/projects/9bd1f0282cf840daacc54b94b0fda178/users/4664852d23ca4d07b67bc208e8237b9f/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.500113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.500527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f4106-2e8c-4a97-8b4e-3cf6cbd37907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd1f0282cf840daacc54b94b0fda178, user_id=4664852d23ca4d07b67bc208e8237b9f, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.501567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.502010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.502421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.503221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.504445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f4106-2e8c-4a97-8b4e-3cf6cbd37907 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.505178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.529707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1065/2112] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:52.529845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1047/2112] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:52 2022] PUT /identity/v3/projects/9bd1f0282cf840daacc54b94b0fda178/users/4664852d23ca4d07b67bc208e8237b9f/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:52.530775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.532042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.532461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.532826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.541896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a02028-579c-4852-8d23-19d0471075d8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.567902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] MFA Rules not processed for user `4664852d23ca4d07b67bc208e8237b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:52.569909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.611668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c870302-ab64-4df0-b3ee-5e50327f7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.612359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c870302-ab64-4df0-b3ee-5e50327f7226 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.612842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c870302-ab64-4df0-b3ee-5e50327f7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.613011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c870302-ab64-4df0-b3ee-5e50327f7226 admin admin] PATH_INFO: `/v3/projects/04812c1d0be74da299712dc3d91f6249/users/4cd6c058741c4f1f906aa441e72cbe5b/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.615108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c870302-ab64-4df0-b3ee-5e50327f7226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04812c1d0be74da299712dc3d91f6249, user_id=4cd6c058741c4f1f906aa441e72cbe5b, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.618727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c870302-ab64-4df0-b3ee-5e50327f7226 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.634379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1066/2113] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:52 2022] PUT /identity/v3/projects/04812c1d0be74da299712dc3d91f6249/users/4cd6c058741c4f1f906aa441e72cbe5b/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:52.645484 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.645813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.645998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.662387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1048/2114] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:52.676488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.678681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.690601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] MFA Rules not processed for user `4cd6c058741c4f1f906aa441e72cbe5b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:52.692086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.734182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:52.754341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a9512-5b85-4bf2-a8e4-c4275e465481 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.773890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1067/2115] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:52.775173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.776826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.806622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.807285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.807546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.807837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.846709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:52.853335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c469-4d0d-469e-a7aa-f7eb94bd6215 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.858294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.858966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.859292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.859562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] PATH_INFO: `/v3/projects/5a5af40474ef4ed09d723d746a56c5fd` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.860709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5af40474ef4ed09d723d746a56c5fd)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.862645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.880304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:52.881748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:52.892760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1049/2116] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:51:52.893718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:52.895323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:52.908237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb3da-18cb-4297-b16b-ee73ce01860c service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:52.925042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5af40474ef4ed09d723d746a56c5fd', 'request_id': 'req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.934390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5af40474ef4ed09d723d746a56c5fd', 'request_id': 'req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.943240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5af40474ef4ed09d723d746a56c5fd', 'request_id': 'req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.963501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] The token cache is being invalidate because project 5a5af40474ef4ed09d723d746a56c5fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:52.963797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.966155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7cabd4c9-38ff-4ea4-ab84-0a5203433d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:52.969946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1068/2117] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:52 2022] DELETE /identity/v3/projects/5a5af40474ef4ed09d723d746a56c5fd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:52.974685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fa40f-622c-4bf2-aa1d-83b7bc209f19 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.975017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fa40f-622c-4bf2-aa1d-83b7bc209f19 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.975233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fa40f-622c-4bf2-aa1d-83b7bc209f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.993479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c5fcd-e28e-42f7-9d86-53e3ec8f23c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:52.994116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c5fcd-e28e-42f7-9d86-53e3ec8f23c6 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:52.994394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c5fcd-e28e-42f7-9d86-53e3ec8f23c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:52.994874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c5fcd-e28e-42f7-9d86-53e3ec8f23c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:52.996878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c15c5fcd-e28e-42f7-9d86-53e3ec8f23c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:53.006981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-010fa40f-622c-4bf2-aa1d-83b7bc209f19 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:53.008480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-010fa40f-622c-4bf2-aa1d-83b7bc209f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:53.065453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c5fcd-e28e-42f7-9d86-53e3ec8f23c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.067399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c5fcd-e28e-42f7-9d86-53e3ec8f23c6 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.074200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:53.078956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1050/2118] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:53.092409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1069/2119] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:53.095851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85f5dd-6ee4-4af4-8b0e-c491e500c4c8 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.096302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85f5dd-6ee4-4af4-8b0e-c491e500c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.096760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85f5dd-6ee4-4af4-8b0e-c491e500c4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.119042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8c85f5dd-6ee4-4af4-8b0e-c491e500c4c8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:53.120192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8c85f5dd-6ee4-4af4-8b0e-c491e500c4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:53.128661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1051/2120] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:53.133244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fa40f-622c-4bf2-aa1d-83b7bc209f19 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.146641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fa40f-622c-4bf2-aa1d-83b7bc209f19 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.154052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.154783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.155074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.155856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.169392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:53.169859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.172204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.190735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1070/2121] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:53.192849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85f5dd-6ee4-4af4-8b0e-c491e500c4c8 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.203408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85f5dd-6ee4-4af4-8b0e-c491e500c4c8 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.210426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.211042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.211324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.211594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.212219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.214368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.247117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1052/2122] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:53 2022] POST /identity/v3/projects => generated 361 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:53.251532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.263100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d732d38-a1be-41f7-b61b-01edc435a8bb admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.309322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.311085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:53.335260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.336822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.337431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.341389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.342322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.344954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.376539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1071/2123] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:53 2022] POST /identity/v3/users => generated 281 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:53.382591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.412834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf6e73-585e-459f-8820-ae04fd6f89a2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.416658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:53.423182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.424517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.425020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.425439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.426340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.428949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.429454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcb260-4bf4-49d1-b11c-12f4907fe04b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.440245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4664852d23ca4d07b67bc208e8237b9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd1f0282cf840daacc54b94b0fda178', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.441041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.441189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1072/2124] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:53.441667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.442091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.442829 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:53.443127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.444758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.454649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.457692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1053/2125] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:51:53.462003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.468098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4803-98f3-4f54-a229-9096dc94d5e0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.473540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.474218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83016dea-0287-4b7a-a884-5bd862dcad53 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.474667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.474946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.475283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.475888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.477535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.479737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4664852d23ca4d07b67bc208e8237b9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd1f0282cf840daacc54b94b0fda178', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.480025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.480463 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.480771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.481632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.483717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.493847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1073/2126] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:53.498046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.510521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b391f-8947-49a1-9160-8146964ade6b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.518472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc897229-a075-4dde-88c3-b4993b485ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.519528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc897229-a075-4dde-88c3-b4993b485ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.519863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc897229-a075-4dde-88c3-b4993b485ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.520075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc897229-a075-4dde-88c3-b4993b485ffb admin admin] PATH_INFO: `/v3/projects/ebe69ce760a34965b730d2a76fc95189/users/1c3dfa398a794e168857fac3d0ed9e0c/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.524306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1054/2127] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:53 2022] POST /identity/v3/projects => generated 352 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:53.529375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.544178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc897229-a075-4dde-88c3-b4993b485ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe69ce760a34965b730d2a76fc95189, user_id=1c3dfa398a794e168857fac3d0ed9e0c, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.549094 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc897229-a075-4dde-88c3-b4993b485ffb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.566059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1074/2128] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:53 2022] PUT /identity/v3/projects/ebe69ce760a34965b730d2a76fc95189/users/1c3dfa398a794e168857fac3d0ed9e0c/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:53.579165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.579396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.579646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.608642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] MFA Rules not processed for user `1c3dfa398a794e168857fac3d0ed9e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:53.609875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:53.629761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9516e6e-8678-4f37-8855-3ae04b4a742a tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.686766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1075/2129] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:53.699934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4664852d23ca4d07b67bc208e8237b9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd1f0282cf840daacc54b94b0fda178', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.700688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.700971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.701246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.703169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:53.703169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.703894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.718217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1055/2130] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:53.725051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.726669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:53.750640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:53.757575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.768041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139844-14df-487b-b105-13261972b782 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.773085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4664852d23ca4d07b67bc208e8237b9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd1f0282cf840daacc54b94b0fda178', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.773822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.774107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.774379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] PATH_INFO: `/v3/projects/b20ad912849f4fde89f491156fade6b8` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.775553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b20ad912849f4fde89f491156fade6b8)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.777660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.788737 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf43042-818a-4f7d-af03-825629d21da9 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.792515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.793165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.793463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.793730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.802667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.804099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:53.814123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1076/2131] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:53.830292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:53.833605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b20ad912849f4fde89f491156fade6b8', 'request_id': 'req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:53.839435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b20ad912849f4fde89f491156fade6b8', 'request_id': 'req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:53.847926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b20ad912849f4fde89f491156fade6b8', 'request_id': 'req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:53.849925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:53.871182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] The token cache is being invalidate because project b20ad912849f4fde89f491156fade6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:53.871475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:53.872722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-695ac4f1-e53e-437e-9ee8-7c7762aef1d5 tempest-VolumeQuotaClassesTest-2114292582 tempest-VolumeQuotaClassesTest-2114292582-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:53.875447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1056/2132] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:53 2022] DELETE /identity/v3/projects/b20ad912849f4fde89f491156fade6b8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:53.918199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392af48-4896-46fc-8206-c5647fa0d66b service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:53.988812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:53.990198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:53.992677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:53.993011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] PATH_INFO: `/v3/users/1c3dfa398a794e168857fac3d0ed9e0c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:53.995156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3dfa398a794e168857fac3d0ed9e0c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:53.997335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:54.052782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3dfa398a794e168857fac3d0ed9e0c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.053414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3dfa398a794e168857fac3d0ed9e0c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.054030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3dfa398a794e168857fac3d0ed9e0c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.062386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3dfa398a794e168857fac3d0ed9e0c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.071316 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3dfa398a794e168857fac3d0ed9e0c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.079177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:54.099581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1077/2133] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:53 2022] DELETE /identity/v3/users/1c3dfa398a794e168857fac3d0ed9e0c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:54.103948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.104249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.104544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.107742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:54.109578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.126911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:54.128122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.187975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8e653-bcc2-4965-ac51-8d3a43699e5c admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:54.192795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:54.193840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.193840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.194031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] PATH_INFO: `/v3/users/7f41e62cd2974352a828df4dd03f3613` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.195116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f41e62cd2974352a828df4dd03f3613)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:54.197288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:54.209110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1057/2134] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:54.222187 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:54.224310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.265033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f41e62cd2974352a828df4dd03f3613'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.265710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f41e62cd2974352a828df4dd03f3613'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.266326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f41e62cd2974352a828df4dd03f3613'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.273968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f41e62cd2974352a828df4dd03f3613'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.282759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-36b2e0f8-7caf-4b18-8a09-6fdc6400c9ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f41e62cd2974352a828df4dd03f3613'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.285445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39b48c-7b22-414e-8fa7-e2a39d290693 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:54.289014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:54.289617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.289904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.290162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.291906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.301220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1078/2135] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:54 2022] DELETE /identity/v3/users/7f41e62cd2974352a828df4dd03f3613 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:54.303844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:54.305335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:54.305592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.305874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.306142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.315194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1058/2136] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:51:54.326215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:54.327501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.398144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1079/2137] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:54.413361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:54.414965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.427575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b13cfd-529b-4bce-9218-e3f0c9bfa7d3 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:54.527631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:54.528295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.528709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.528973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.539809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:54.541328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:54.552054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1059/2138] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:54.652618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:54.654815 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.674682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612db3f6-f04f-49ab-9224-db74dd9865ab None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:54.695909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:54.696804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.697220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.697567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] PATH_INFO: `/v3/projects/ebe69ce760a34965b730d2a76fc95189` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.702944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe69ce760a34965b730d2a76fc95189)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:54.702944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:54.703862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:54.738141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:54.740315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.797153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe69ce760a34965b730d2a76fc95189', 'request_id': 'req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.797781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:54.808943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe69ce760a34965b730d2a76fc95189', 'request_id': 'req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.825992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe69ce760a34965b730d2a76fc95189', 'request_id': 'req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.829494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75453c1-77a5-4f6d-8020-49545bca9d67 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:54.847385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] The token cache is being invalidate because project ebe69ce760a34965b730d2a76fc95189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:54.847842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.848611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2452cd47-934a-4a9a-b4e6-8f76d53c0b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:54.852308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1080/2139] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:54 2022] DELETE /identity/v3/projects/ebe69ce760a34965b730d2a76fc95189 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:54.855532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabf44e-b114-429d-bd15-f02121e3b452 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.855915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabf44e-b114-429d-bd15-f02121e3b452 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.857865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabf44e-b114-429d-bd15-f02121e3b452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.885039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ceabf44e-b114-429d-bd15-f02121e3b452 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:54.887330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ceabf44e-b114-429d-bd15-f02121e3b452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:54.941323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:54.942668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:54.942947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:54.943219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:54.947399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.014369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1081/2140] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:55.022600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.022887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.023163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.056718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:55.058016 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.065039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.066825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.072044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1082/2141] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:55.073588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.074407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.079442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1060/2142] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:55.080336 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.086277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04c86-60ae-49c5-8c73-90058e3f7c85 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.091338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.091975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.092247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.092739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] PATH_INFO: `/v3/projects/0cd4591fd4944500b0697cb9abed7247` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.093540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f8e7a-d3a4-4e84-bd4a-aa1a67a0fbe6 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.093875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd4591fd4944500b0697cb9abed7247)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.095774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.098852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.099574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.099937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.100295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.101029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:55.101387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.103293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.117562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1061/2143] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:55.122941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.133849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633315e-8551-4aab-a7be-a6ac75c7362a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.140554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.141213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.141470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.141741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.142366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.144773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.185828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd4591fd4944500b0697cb9abed7247', 'request_id': 'req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.192135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1062/2144] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:55 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:55.196402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.198511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd4591fd4944500b0697cb9abed7247', 'request_id': 'req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.205923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd4591fd4944500b0697cb9abed7247', 'request_id': 'req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.223970 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] The token cache is being invalidate because project 0cd4591fd4944500b0697cb9abed7247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:55.224297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.224972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.228369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1083/2145] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:55 2022] DELETE /identity/v3/projects/0cd4591fd4944500b0697cb9abed7247 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:55.230890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.232249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.243672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:55.287021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d896cc-428e-4094-8cc0-13b053fe3ff7 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.296626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.297717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.298119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.298751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.299492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.303158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.317792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9833fa-3db9-4e00-b1ae-3c4eb557c146 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.331335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.331999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.340467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.340467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] PATH_INFO: `/v3/users/4664852d23ca4d07b67bc208e8237b9f` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.340467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4664852d23ca4d07b67bc208e8237b9f)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.340467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.340895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1063/2146] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:55 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:55.342448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.344895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.353791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:55.365056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111e96e-912e-4cb4-ae3b-8f6a1eaa3d73 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.373904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.374658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.374969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.375234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.376006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.377941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.391425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1064/2147] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:55.397080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.409149 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ea9be-b29e-46b0-820a-022ba236d04f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.411011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4664852d23ca4d07b67bc208e8237b9f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.411924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4664852d23ca4d07b67bc208e8237b9f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.412663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4664852d23ca4d07b67bc208e8237b9f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.414069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.414681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.414956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.415227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.415877 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.417627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.423116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4664852d23ca4d07b67bc208e8237b9f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.434823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-26205c1b-95df-42fe-8af8-e56c5a2bcff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4664852d23ca4d07b67bc208e8237b9f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:55.438035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1065/2148] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:55.443156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.446856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.469266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da529-930f-4536-9c72-0e8030f215eb admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.469394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1084/2149] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:51:55 2022] DELETE /identity/v3/users/4664852d23ca4d07b67bc208e8237b9f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:55.556685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.557490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.557705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.558294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] PATH_INFO: `/v3/projects/d54ff82d86764c05b1b1bcf200df6741/users/c770ab1dc73a4e5eb3176d33100f9084/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.574562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54ff82d86764c05b1b1bcf200df6741, user_id=c770ab1dc73a4e5eb3176d33100f9084, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.577190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.612319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1066/2150] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:55 2022] PUT /identity/v3/projects/d54ff82d86764c05b1b1bcf200df6741/users/c770ab1dc73a4e5eb3176d33100f9084/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:55.618895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.619168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.619503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.653010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] MFA Rules not processed for user `c770ab1dc73a4e5eb3176d33100f9084`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:55.654538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.781914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1085/2151] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:55.786073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.844533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.846114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.874491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:55.891315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234d3b-413f-4e96-ab38-0b659d0c1276 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:55.898442 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:55.899419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:55.899742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:55.900012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:55.900935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:55.903711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:55.960624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1067/2152] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:55 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:55.961691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:55.963583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:55.986054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7808a-e6d1-41ae-ae4f-5669b361c6de None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.028300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:56.033001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa0638-3d18-4a56-be01-1549db041d24 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.037554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.038404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.038916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.039207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.041150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:56.047137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.047754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.048031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.048332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] PATH_INFO: `/v3/projects/9bd1f0282cf840daacc54b94b0fda178` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.049444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd1f0282cf840daacc54b94b0fda178)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.051451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.105211 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd1f0282cf840daacc54b94b0fda178', 'request_id': 'req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:56.109475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.111301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.112112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd1f0282cf840daacc54b94b0fda178', 'request_id': 'req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:56.120874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd1f0282cf840daacc54b94b0fda178', 'request_id': 'req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:56.124146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1068/2153] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:56.125094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.138360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ca8cf-284e-47d6-91c7-7595deab87ec service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.145127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] The token cache is being invalidate because project 9bd1f0282cf840daacc54b94b0fda178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:51:56.145436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:56.146164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-5fd1a78f-1e62-4eb1-a8cf-58ab66dbc329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:51:56.147968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1086/2154] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:51:55 2022] DELETE /identity/v3/projects/9bd1f0282cf840daacc54b94b0fda178 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:56.153620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be124-7b54-4cda-af80-f9785f98032f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.153620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be124-7b54-4cda-af80-f9785f98032f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.153864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be124-7b54-4cda-af80-f9785f98032f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.187915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-781be124-7b54-4cda-af80-f9785f98032f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:56.190011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-781be124-7b54-4cda-af80-f9785f98032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:56.230548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.231198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.231475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.231744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.232403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.234438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.276303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1069/2155] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:56.280040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.282026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:56.301338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:56.306554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db77a56-d264-4129-9e06-c785689b2015 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.311611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1087/2156] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:56.312757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.313375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.313668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.314155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.315598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.317598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.320536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191af52-c0f7-4235-b9a2-0ae9a8de3b60 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.321027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191af52-c0f7-4235-b9a2-0ae9a8de3b60 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.321473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191af52-c0f7-4235-b9a2-0ae9a8de3b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.345305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1070/2157] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:56.347964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f191af52-c0f7-4235-b9a2-0ae9a8de3b60 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:56.349341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f191af52-c0f7-4235-b9a2-0ae9a8de3b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:56.352197 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.361876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1088/2158] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:56.363004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191af52-c0f7-4235-b9a2-0ae9a8de3b60 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.366085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca4505-9cd9-40e1-affb-0498e77fbb10 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.372131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.374644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191af52-c0f7-4235-b9a2-0ae9a8de3b60 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.381171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.381791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.382069 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.382340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.383078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:51:56.383279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.385402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.386139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.386426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.390068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.390960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.399668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1089/2159] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:51:56.404061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.409531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.416056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0354d0-0404-44e1-b037-e525d17e7c35 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.422461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.423080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.423361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.423636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.423934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1071/2160] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:56.425400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.426762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.428497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.439326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b350771-09f4-4420-b5a2-f16ec37d1e2a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.450337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f810-21f3-43c8-b37f-225c6db88483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.451014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f810-21f3-43c8-b37f-225c6db88483 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.451309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f810-21f3-43c8-b37f-225c6db88483 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.451656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f810-21f3-43c8-b37f-225c6db88483 admin admin] PATH_INFO: `/v3/projects/d7de6c6c59024d6582615f23d862e1c9/users/fa968ed5a86f4c088490a3965316afc8/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.465599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1090/2161] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:56.471328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.477928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f810-21f3-43c8-b37f-225c6db88483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7de6c6c59024d6582615f23d862e1c9, user_id=fa968ed5a86f4c088490a3965316afc8, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.483922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f810-21f3-43c8-b37f-225c6db88483 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.499087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1072/2162] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:56 2022] PUT /identity/v3/projects/d7de6c6c59024d6582615f23d862e1c9/users/fa968ed5a86f4c088490a3965316afc8/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:56.511085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c81a1-9aa6-4300-98f3-ef14d4491d29 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.511446 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c81a1-9aa6-4300-98f3-ef14d4491d29 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.511783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c81a1-9aa6-4300-98f3-ef14d4491d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.543419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-5f9c81a1-9aa6-4300-98f3-ef14d4491d29 None None] MFA Rules not processed for user `fa968ed5a86f4c088490a3965316afc8`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:56.544860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5f9c81a1-9aa6-4300-98f3-ef14d4491d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:56.592103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec54f8-0904-4cfd-907d-089e11245465 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.602275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:56.648864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1073/2163] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:56.653952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c81a1-9aa6-4300-98f3-ef14d4491d29 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.692526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.693452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.693876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.694383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.695247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.697319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.727155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1091/2164] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/users => generated 296 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:56.731983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.745964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5332029-8ba2-4280-aed5-e17a559f5a57 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.750874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.751507 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.751753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.752034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.752868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.754491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.764642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c81a1-9aa6-4300-98f3-ef14d4491d29 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.767345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1092/2165] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:56.771127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.771677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.771887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.772161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.772487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.773113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.775014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.782204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31756f40-152e-4345-843b-cfea0500b4c7 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.788199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.789343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.789727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.789998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.791459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.792773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.805732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1093/2166] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:56.810350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.812604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1074/2167] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:56.814705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.821570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345266bf-423b-43f4-8e4b-3ccb6a22ab10 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.854844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:56.892767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc7733-4a0d-4397-8515-8c6d80ecb367 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.898358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.898958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.899230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.899507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.900124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.900810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebc5e7-3ab0-4103-85c2-82dcef2ef036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:56.901709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebc5e7-3ab0-4103-85c2-82dcef2ef036 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.901984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebc5e7-3ab0-4103-85c2-82dcef2ef036 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.902343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.902715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebc5e7-3ab0-4103-85c2-82dcef2ef036 admin admin] PATH_INFO: `/v3/projects/5bcf9c2e53b8493faa2c527232d5ffe7/users/22e8c623af8f426c98fa25f496a38c95/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:56.931840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebc5e7-3ab0-4103-85c2-82dcef2ef036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bcf9c2e53b8493faa2c527232d5ffe7, user_id=22e8c623af8f426c98fa25f496a38c95, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:56.935474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebc5e7-3ab0-4103-85c2-82dcef2ef036 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:56.938883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1075/2168] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:51:56.942718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:56.956404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f72be9-e072-49a4-9d03-c2b9b9582392 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:56.960338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1094/2169] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:56 2022] PUT /identity/v3/projects/5bcf9c2e53b8493faa2c527232d5ffe7/users/22e8c623af8f426c98fa25f496a38c95/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:56.967357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3639ba-09ad-45e1-8305-4847668f4c94 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:56.967635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3639ba-09ad-45e1-8305-4847668f4c94 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:56.967899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3639ba-09ad-45e1-8305-4847668f4c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.016282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-fd3639ba-09ad-45e1-8305-4847668f4c94 None None] MFA Rules not processed for user `22e8c623af8f426c98fa25f496a38c95`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:57.017715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-fd3639ba-09ad-45e1-8305-4847668f4c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:57.022362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:57.039021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.039686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.040093 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.040405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.041058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.042953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.053167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1076/2170] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:57.057346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.067351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33fb3-3cde-4727-9ddc-fe68e9762b1d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.072354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.073085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.073373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.073613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] PATH_INFO: `/v3/projects/2882aa531f3c4c539914592952dd5571/users/39f569069bbb42eeab4d6fc6977c10fa/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.086933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1095/2171] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:57.089125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2882aa531f3c4c539914592952dd5571, user_id=39f569069bbb42eeab4d6fc6977c10fa, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.091637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3639ba-09ad-45e1-8305-4847668f4c94 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.091865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.102011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3639ba-09ad-45e1-8305-4847668f4c94 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.105725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1077/2172] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:57 2022] PUT /identity/v3/projects/2882aa531f3c4c539914592952dd5571/users/39f569069bbb42eeab4d6fc6977c10fa/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:57.109770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.164747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.165354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.165624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.165988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.166729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.168664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.172511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411972d-2ee0-4695-8ff8-4361ca9c4342 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.177998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.178598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.178870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.179267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.179911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.181617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.193498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1078/2173] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:57.198364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.201962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1096/2174] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:51:57 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:57.206280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.210903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c30bcd-6b9b-498f-b43f-0f1e01e58f00 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.239121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:57.275313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e06a0c-91b2-490f-b320-2b5bd012445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.276935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06a0c-91b2-490f-b320-2b5bd012445f admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.277291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06a0c-91b2-490f-b320-2b5bd012445f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.277624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06a0c-91b2-490f-b320-2b5bd012445f admin admin] PATH_INFO: `/v3/projects/2882aa531f3c4c539914592952dd5571/users/39f569069bbb42eeab4d6fc6977c10fa/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.277729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75532eaa-95dd-45a1-b1fa-82d8ce6048cd admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.279655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e06a0c-91b2-490f-b320-2b5bd012445f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2882aa531f3c4c539914592952dd5571, user_id=39f569069bbb42eeab4d6fc6977c10fa, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.282560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e06a0c-91b2-490f-b320-2b5bd012445f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.282690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.283224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.283494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.284018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.284403 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.286296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.298630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1079/2175] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:57 2022] PUT /identity/v3/projects/2882aa531f3c4c539914592952dd5571/users/39f569069bbb42eeab4d6fc6977c10fa/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:51:57.311103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1097/2176] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:51:57 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:51:57.312988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.313270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.313536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.314722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.325520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c5207-a01a-49b0-9bda-86bc02a3f6b0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.354779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] MFA Rules not processed for user `39f569069bbb42eeab4d6fc6977c10fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:57.356228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:57.400169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.401075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.401672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.401908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.402594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:57.403465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.404874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.418343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1098/2177] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:51:57.419857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.429861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f17e47-d4ff-437a-a0a8-57a661dbb9fb admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.435112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.435778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.436033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.436322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] PATH_INFO: `/v3/projects/4993a7d5b4de4358a2f52759ce10101d/users/71fdab5bba6a4f57af4e18ddd68eacc1/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.442443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1080/2178] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:51:57.452311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4993a7d5b4de4358a2f52759ce10101d, user_id=71fdab5bba6a4f57af4e18ddd68eacc1, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.455074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.455706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.457286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:57.470931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1099/2179] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:57 2022] PUT /identity/v3/projects/4993a7d5b4de4358a2f52759ce10101d/users/71fdab5bba6a4f57af4e18ddd68eacc1/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:57.474743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.537982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72b813-8150-47ea-8822-faa1fe52ae4a None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.555916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed6218-a94c-4ac8-b866-1d07615acb99 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.561555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.562174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.562444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.562829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.563621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.565407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.577570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1100/2180] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:51:57.581392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.594874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd310-72e2-4244-a0e3-d10a4ba648a2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.608474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04638be6-94de-4eb7-85c7-40c3b8068902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.608474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04638be6-94de-4eb7-85c7-40c3b8068902 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.608474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04638be6-94de-4eb7-85c7-40c3b8068902 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.608474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04638be6-94de-4eb7-85c7-40c3b8068902 admin admin] PATH_INFO: `/v3/projects/4993a7d5b4de4358a2f52759ce10101d/users/71fdab5bba6a4f57af4e18ddd68eacc1/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.608852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04638be6-94de-4eb7-85c7-40c3b8068902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4993a7d5b4de4358a2f52759ce10101d, user_id=71fdab5bba6a4f57af4e18ddd68eacc1, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.608852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04638be6-94de-4eb7-85c7-40c3b8068902 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.614411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.615102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.615722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.615722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.617448 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:57.621740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1101/2181] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:51:57 2022] PUT /identity/v3/projects/4993a7d5b4de4358a2f52759ce10101d/users/71fdab5bba6a4f57af4e18ddd68eacc1/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:51:57.632311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7934a8-08c3-4a34-97ee-c6370bb86e85 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.632754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7934a8-08c3-4a34-97ee-c6370bb86e85 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.633090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7934a8-08c3-4a34-97ee-c6370bb86e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.658950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:57.670598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ce7934a8-08c3-4a34-97ee-c6370bb86e85 None None] MFA Rules not processed for user `71fdab5bba6a4f57af4e18ddd68eacc1`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:51:57.672808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ce7934a8-08c3-4a34-97ee-c6370bb86e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:57.719936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.721439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.730656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1081/2182] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:57.748378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1102/2183] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:51:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:51:57.759619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.761134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:51:57.772245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75058a8-cc75-4bd6-a553-8350d825421f service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.818591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:51:57.829909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.830520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.831461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.831711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.841238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.842696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:57.852810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1082/2184] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:51:57.903529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7934a8-08c3-4a34-97ee-c6370bb86e85 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:57.913635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7934a8-08c3-4a34-97ee-c6370bb86e85 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:57.921014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:57.921595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:57.921958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:57.922154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:57.989572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:57.991108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:58.008674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1103/2185] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:51:59.332357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:51:59.355566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c272f-bf63-491f-8d2c-25144130ddb2 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:51:59.364886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:51:59.365558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:51:59.365823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:51:59.366089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:51:59.537838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:51:59.539568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:51:59.551440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1083/2186] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:51:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:00.194407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:00.205701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4eff6-fcfe-48ad-b894-766d1e6cc6b4 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:00.211642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91151b7c-7b98-4613-b12e-7b60fe1210f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:00.212621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151b7c-7b98-4613-b12e-7b60fe1210f8 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:00.213255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151b7c-7b98-4613-b12e-7b60fe1210f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:00.213638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151b7c-7b98-4613-b12e-7b60fe1210f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:00.300544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91151b7c-7b98-4613-b12e-7b60fe1210f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:00.302184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91151b7c-7b98-4613-b12e-7b60fe1210f8 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:00.314745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1104/2187] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:00.338361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:00.338361 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:00.430813 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0500e-8032-4f38-af80-e936c69bce31 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:00.435731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:00.436394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:00.437003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:00.437418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] PATH_INFO: `/v3/users/29aea39f1249420c8585c1685d993dcd` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:00.438700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29aea39f1249420c8585c1685d993dcd)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:00.441015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:00.501604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29aea39f1249420c8585c1685d993dcd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:00.511946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29aea39f1249420c8585c1685d993dcd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:00.512748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29aea39f1249420c8585c1685d993dcd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:00.524883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29aea39f1249420c8585c1685d993dcd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:00.543039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29aea39f1249420c8585c1685d993dcd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:00.549289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1084/2188] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:00 2022] DELETE /identity/v3/users/29aea39f1249420c8585c1685d993dcd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:00.554285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:00.554556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:00.554887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:00.574796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:00.576042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:00.601543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:00.638851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1105/2189] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:00.649055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:00.650676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:00.657839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:00.715087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1500f5-4d42-42f3-b8ef-fb2cd8733264 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:00.718901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:00.719505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:00.719776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:00.720042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:00.729722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:00.731162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:00.742294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1085/2190] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:01.091416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:01.092913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:01.108363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3ebbc-ac98-40e0-ab79-fbf806dc90d6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:01.117629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:01.118558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:01.118899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:01.119335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] PATH_INFO: `/v3/projects/37fcea6a1e024829bf68678aa5b3a263` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:01.120928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37fcea6a1e024829bf68678aa5b3a263)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:01.122865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:01.173398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37fcea6a1e024829bf68678aa5b3a263', 'request_id': 'req-f799a640-345b-464d-873b-534d518f2e56'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.180154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37fcea6a1e024829bf68678aa5b3a263', 'request_id': 'req-f799a640-345b-464d-873b-534d518f2e56'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.187622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37fcea6a1e024829bf68678aa5b3a263', 'request_id': 'req-f799a640-345b-464d-873b-534d518f2e56'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.201621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] The token cache is being invalidate because project 37fcea6a1e024829bf68678aa5b3a263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:01.202192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.202509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.204011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1106/2191] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:01 2022] DELETE /identity/v3/projects/37fcea6a1e024829bf68678aa5b3a263 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:01.208012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:01.218133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b38b-1f85-4a57-8e59-2266db6c409c service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:01.276485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:01.277104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:01.277398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:01.277668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] PATH_INFO: `/v3/users/4edf938227b94bc1982f581e52d6c8b5` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:01.278834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4edf938227b94bc1982f581e52d6c8b5)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:01.280810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:01.337105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4edf938227b94bc1982f581e52d6c8b5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.349166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4edf938227b94bc1982f581e52d6c8b5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.349999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4edf938227b94bc1982f581e52d6c8b5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.358549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4edf938227b94bc1982f581e52d6c8b5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.372082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4edf938227b94bc1982f581e52d6c8b5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.378691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1086/2192] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:01 2022] DELETE /identity/v3/users/4edf938227b94bc1982f581e52d6c8b5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:01.680012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:01.682492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:01.694094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:01.696558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:01.757059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a640-345b-464d-873b-534d518f2e56 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:01.761690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:01.832514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000699e5-973d-437c-b356-33a710315a43 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:01.849021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:01.849786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:01.850225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:01.850736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] PATH_INFO: `/v3/projects/6a9a05818ab64b8fb6de685453b2dad8` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:01.852775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a9a05818ab64b8fb6de685453b2dad8)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:01.855116 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:01.912773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:01.913879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:01.914306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:01.917663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] PATH_INFO: `/v3/users/22e8c623af8f426c98fa25f496a38c95` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:01.921047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e8c623af8f426c98fa25f496a38c95)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:01.925635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:01.961464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a9a05818ab64b8fb6de685453b2dad8', 'request_id': 'req-89e4016d-89f9-4da0-9f37-f177fe20946b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.973430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a9a05818ab64b8fb6de685453b2dad8', 'request_id': 'req-89e4016d-89f9-4da0-9f37-f177fe20946b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:01.994188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a9a05818ab64b8fb6de685453b2dad8', 'request_id': 'req-89e4016d-89f9-4da0-9f37-f177fe20946b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.024814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] The token cache is being invalidate because project 6a9a05818ab64b8fb6de685453b2dad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:02.025643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.027404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.030796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1087/2193] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:01 2022] DELETE /identity/v3/projects/6a9a05818ab64b8fb6de685453b2dad8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:02.037454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:02.040329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:02.072474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e8c623af8f426c98fa25f496a38c95'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.074198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e8c623af8f426c98fa25f496a38c95'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.077666 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e8c623af8f426c98fa25f496a38c95'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.088638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:02.091542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e8c623af8f426c98fa25f496a38c95'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.106083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b55df948-7eab-4ea0-ba34-89ed766745aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e8c623af8f426c98fa25f496a38c95'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.138776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1107/2194] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:01 2022] DELETE /identity/v3/users/22e8c623af8f426c98fa25f496a38c95 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:02.143432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4016d-89f9-4da0-9f37-f177fe20946b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:02.145241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:02.145537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:02.145829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:02.174319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:02.175733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:02.228933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:02.234653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:02.234983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:02.235261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] PATH_INFO: `/v3/users/4cd6c058741c4f1f906aa441e72cbe5b` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:02.237923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd6c058741c4f1f906aa441e72cbe5b)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:02.238948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:02.245883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:02.273716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1108/2195] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:02.283954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:02.285576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:02.327386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd6c058741c4f1f906aa441e72cbe5b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.336761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd6c058741c4f1f906aa441e72cbe5b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.338614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd6c058741c4f1f906aa441e72cbe5b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.364529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd6c058741c4f1f906aa441e72cbe5b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.366582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd6c058741c4f1f906aa441e72cbe5b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.374226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1088/2196] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:02 2022] DELETE /identity/v3/users/4cd6c058741c4f1f906aa441e72cbe5b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:02.380103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fd104-e08b-4709-8701-3bf5d0055724 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:02.437855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:02.438495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:02.438731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:02.438992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:02.440954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:02.496854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:02.533052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:02.534774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:02.550448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1109/2197] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:52:02.770259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:02.771773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:02.786426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29be1e-c3a4-46b5-8ad9-53eaf3ed0bfd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:02.792561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:02.793232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:02.793504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:02.793765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] PATH_INFO: `/v3/projects/04812c1d0be74da299712dc3d91f6249` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:02.795018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04812c1d0be74da299712dc3d91f6249)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:02.797021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:02.853128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04812c1d0be74da299712dc3d91f6249', 'request_id': 'req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.860303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04812c1d0be74da299712dc3d91f6249', 'request_id': 'req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.867823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04812c1d0be74da299712dc3d91f6249', 'request_id': 'req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.890335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] The token cache is being invalidate because project 04812c1d0be74da299712dc3d91f6249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:02.890633 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.891308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:02.892980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1089/2198] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:02 2022] DELETE /identity/v3/projects/04812c1d0be74da299712dc3d91f6249 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:02.948074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:02.953540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:02.973980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d26ad-e632-45bd-af73-aa759254ce7e service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:03.084419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:03.086199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:03.086634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:03.087221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] PATH_INFO: `/v3/projects/5bcf9c2e53b8493faa2c527232d5ffe7` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:03.092768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bcf9c2e53b8493faa2c527232d5ffe7)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:03.094945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:03.165299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bcf9c2e53b8493faa2c527232d5ffe7', 'request_id': 'req-af976958-04c4-4193-8f5d-d55bfe4a9754'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.171738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bcf9c2e53b8493faa2c527232d5ffe7', 'request_id': 'req-af976958-04c4-4193-8f5d-d55bfe4a9754'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.179775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bcf9c2e53b8493faa2c527232d5ffe7', 'request_id': 'req-af976958-04c4-4193-8f5d-d55bfe4a9754'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.194255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] The token cache is being invalidate because project 5bcf9c2e53b8493faa2c527232d5ffe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:03.194548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.195174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.196797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1110/2199] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:02 2022] DELETE /identity/v3/projects/5bcf9c2e53b8493faa2c527232d5ffe7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:03.202101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:03.203859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:03.217152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceaf31-6405-480e-b0f1-82a8f0eb2f63 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:03.277204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:03.277992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:03.278248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:03.278532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] PATH_INFO: `/v3/users/71fdab5bba6a4f57af4e18ddd68eacc1` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:03.279714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] RBAC: Authorizing `identity:delete_user(user_id=71fdab5bba6a4f57af4e18ddd68eacc1)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:03.281725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:03.288175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:03.336589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71fdab5bba6a4f57af4e18ddd68eacc1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.343839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71fdab5bba6a4f57af4e18ddd68eacc1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.344564 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71fdab5bba6a4f57af4e18ddd68eacc1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.359362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71fdab5bba6a4f57af4e18ddd68eacc1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.374658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ca27441b-7b08-4daf-8a08-dcb26cab981f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71fdab5bba6a4f57af4e18ddd68eacc1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.381302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1090/2200] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:03 2022] DELETE /identity/v3/users/71fdab5bba6a4f57af4e18ddd68eacc1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:03.751201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:03.752748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:03.772823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af976958-04c4-4193-8f5d-d55bfe4a9754 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:03.829481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:03.830099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:03.830364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:03.830628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] PATH_INFO: `/v3/projects/4993a7d5b4de4358a2f52759ce10101d` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:03.831745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4993a7d5b4de4358a2f52759ce10101d)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:03.833671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:03.844240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:03.880586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4993a7d5b4de4358a2f52759ce10101d', 'request_id': 'req-2352cb20-773f-4f58-a626-8effdd91d62a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.886858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4993a7d5b4de4358a2f52759ce10101d', 'request_id': 'req-2352cb20-773f-4f58-a626-8effdd91d62a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.892340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4993a7d5b4de4358a2f52759ce10101d', 'request_id': 'req-2352cb20-773f-4f58-a626-8effdd91d62a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.906667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] The token cache is being invalidate because project 4993a7d5b4de4358a2f52759ce10101d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:03.906947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.907553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2352cb20-773f-4f58-a626-8effdd91d62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:03.909064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1111/2201] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:03 2022] DELETE /identity/v3/projects/4993a7d5b4de4358a2f52759ce10101d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:03.916056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cc193-626b-4326-8998-c793c23ab01e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:03.916492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cc193-626b-4326-8998-c793c23ab01e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:03.917656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cc193-626b-4326-8998-c793c23ab01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:03.943575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-291cc193-626b-4326-8998-c793c23ab01e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:03.944819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-291cc193-626b-4326-8998-c793c23ab01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:04.014225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1091/2202] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:04.022923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cccc9-2ed7-488e-8e42-637b52245351 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.023103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cccc9-2ed7-488e-8e42-637b52245351 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.023400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cccc9-2ed7-488e-8e42-637b52245351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.046081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cf5cccc9-2ed7-488e-8e42-637b52245351 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:04.047287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cf5cccc9-2ed7-488e-8e42-637b52245351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:04.057608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1112/2203] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:04.062278 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cc193-626b-4326-8998-c793c23ab01e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.075363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cc193-626b-4326-8998-c793c23ab01e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.080732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.081347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.081619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.081888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.082545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:52:04.082842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.084480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.097462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1092/2204] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:52:04.102055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cccc9-2ed7-488e-8e42-637b52245351 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.112588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cccc9-2ed7-488e-8e42-637b52245351 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.118603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.119205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.119533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.120196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.120478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.122383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.151961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1113/2205] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:52:04 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:04.156493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.167208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da679ff0-c205-4c87-aaef-62c4b7f30d46 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.229784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.230389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.230662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.230933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.231562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.233539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.259220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1093/2206] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:52:04 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:52:04.260541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.272058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baac25a-806a-4738-912c-28841a25d8f4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.277654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.278238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.278476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.278760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.279457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.281065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.291306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1114/2207] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:04.295528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.300731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:04.307287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906f84d-d54c-4141-943d-c603abd8fafd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.313358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef5afd-c49a-416b-9ef0-952b9989fe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.313968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef5afd-c49a-416b-9ef0-952b9989fe89 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.314222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef5afd-c49a-416b-9ef0-952b9989fe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.314491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef5afd-c49a-416b-9ef0-952b9989fe89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.315133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef5afd-c49a-416b-9ef0-952b9989fe89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.317212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef5afd-c49a-416b-9ef0-952b9989fe89 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.329966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1094/2208] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:04.334334 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.344963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51966-a1b3-4b04-a970-46fad694bdab admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.349996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.350644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.350927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.351210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] PATH_INFO: `/v3/projects/20bd93e4f2224aa9877b583b879b2e18/users/5b27e079167b499a975f2490a7d12a8c/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.366343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bd93e4f2224aa9877b583b879b2e18, user_id=5b27e079167b499a975f2490a7d12a8c, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.369036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.384350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1115/2209] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:04 2022] PUT /identity/v3/projects/20bd93e4f2224aa9877b583b879b2e18/users/5b27e079167b499a975f2490a7d12a8c/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:04.396311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d5e77-7881-40dc-962f-addf55ac5a9f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.396769 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d5e77-7881-40dc-962f-addf55ac5a9f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.396936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d5e77-7881-40dc-962f-addf55ac5a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.425836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-139d5e77-7881-40dc-962f-addf55ac5a9f None None] MFA Rules not processed for user `5b27e079167b499a975f2490a7d12a8c`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:04.427113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-139d5e77-7881-40dc-962f-addf55ac5a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:04.500676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1095/2210] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:52:04.505242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.571995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc20eaa-d794-427e-9df1-7859d242461b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.577465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.578147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.578424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.578721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.579342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.581279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.612639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1116/2211] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:52:04 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:04.617106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d5e77-7881-40dc-962f-addf55ac5a9f None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.628023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d5e77-7881-40dc-962f-addf55ac5a9f None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.690671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.691367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.691654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.691921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.692571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.694486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.715322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1096/2212] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:52:04 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:52:04.719301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.729771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78654a4-e188-4cd0-8527-d3ad9eb7248b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.734660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.735314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.735564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.735827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.736497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.738083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.750267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1117/2213] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:04.754563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.765006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bba39a-b6bf-4e55-b497-45c45c884ef1 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.770801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.771445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.771717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.771982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] PATH_INFO: `/v3/projects/71d72c114dd545459024b05d1f5dcf62/users/d96976d475824de28b978a80c8e095a0/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.787697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d72c114dd545459024b05d1f5dcf62, user_id=d96976d475824de28b978a80c8e095a0, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.790374 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.806334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1097/2214] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:04 2022] PUT /identity/v3/projects/71d72c114dd545459024b05d1f5dcf62/users/d96976d475824de28b978a80c8e095a0/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:04.811790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.822156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebf2b5-d338-4aaa-bf43-19b01f7a46d0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.827600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:04.882706 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610ec8d-9c77-4641-939c-600dc85de050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.883308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610ec8d-9c77-4641-939c-600dc85de050 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.883584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610ec8d-9c77-4641-939c-600dc85de050 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.883856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610ec8d-9c77-4641-939c-600dc85de050 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.884525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610ec8d-9c77-4641-939c-600dc85de050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.886313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610ec8d-9c77-4641-939c-600dc85de050 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.896425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1118/2215] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:04.901109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:04.912078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c74b71-0ad6-41c5-8679-0f4c570d5dea admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:04.918104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:04.918922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.919191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.919458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] PATH_INFO: `/v3/projects/71d72c114dd545459024b05d1f5dcf62/users/d96976d475824de28b978a80c8e095a0/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.921645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d72c114dd545459024b05d1f5dcf62, user_id=d96976d475824de28b978a80c8e095a0, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:04.924315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:04.935520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1098/2216] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:04 2022] PUT /identity/v3/projects/71d72c114dd545459024b05d1f5dcf62/users/d96976d475824de28b978a80c8e095a0/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:04.945870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:04.946154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:04.946422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:04.975358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] MFA Rules not processed for user `d96976d475824de28b978a80c8e095a0`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:04.976608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:05.049446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1119/2217] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:52:05.061367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:05.063298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:05.101582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:05.130271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f90d6c-15bf-4606-91fa-9d4ae4816d8e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:05.135662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:05.136301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:05.136580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:05.136846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:05.147643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:05.149208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:05.159161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1099/2218] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:05.248965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:05.250378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:05.262636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7156b49-1420-4454-9359-723bc966efa3 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:05.267044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb2282-b387-4cf4-bc71-295617dd70cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:05.267693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb2282-b387-4cf4-bc71-295617dd70cb service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:05.267992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb2282-b387-4cf4-bc71-295617dd70cb service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:05.268226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb2282-b387-4cf4-bc71-295617dd70cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:05.334630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb2282-b387-4cf4-bc71-295617dd70cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:05.336049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb2282-b387-4cf4-bc71-295617dd70cb service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:05.346248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1120/2219] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:05.648470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:05.714346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f893b6-e071-48d8-b8d9-d968c3239ea6 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:05.719798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:05.720467 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:05.720735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:05.720998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] PATH_INFO: `/v3/users/5b27e079167b499a975f2490a7d12a8c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:05.722223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b27e079167b499a975f2490a7d12a8c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:05.724166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:05.776403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b27e079167b499a975f2490a7d12a8c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:05.785839 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b27e079167b499a975f2490a7d12a8c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:05.786729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b27e079167b499a975f2490a7d12a8c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:05.795862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b27e079167b499a975f2490a7d12a8c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:05.809183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b27e079167b499a975f2490a7d12a8c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:05.815556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1100/2220] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:05 2022] DELETE /identity/v3/users/5b27e079167b499a975f2490a7d12a8c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:05.820125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:05.820669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:05.820762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:05.839529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:05.840786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:05.865137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:05.905582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1121/2221] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:05.915610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:05.917265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:05.986150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca153a7-448b-4d9a-8907-c94a6cbe719f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:05.990631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:05.991228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:05.991504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:05.991797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.001953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:06.003396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:06.007198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:06.014313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1101/2222] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:06.300634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:06.300634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:06.315528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56243f90-ff4e-4c74-bf3d-15f1d0e719cc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:06.321823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:06.322477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.322766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.323026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] PATH_INFO: `/v3/projects/20bd93e4f2224aa9877b583b879b2e18` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.324154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bd93e4f2224aa9877b583b879b2e18)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:06.326132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:06.379142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bd93e4f2224aa9877b583b879b2e18', 'request_id': 'req-1681803d-6136-468c-89ad-3479faa58d1d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.385417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bd93e4f2224aa9877b583b879b2e18', 'request_id': 'req-1681803d-6136-468c-89ad-3479faa58d1d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.391274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bd93e4f2224aa9877b583b879b2e18', 'request_id': 'req-1681803d-6136-468c-89ad-3479faa58d1d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.406360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] The token cache is being invalidate because project 20bd93e4f2224aa9877b583b879b2e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:06.406644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.407349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.408833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1122/2223] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:06 2022] DELETE /identity/v3/projects/20bd93e4f2224aa9877b583b879b2e18 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:06.412862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:06.423049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac2e0-387c-4ef0-865e-5e4baa7e4229 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:06.481139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:06.481756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.482027 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.482292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] PATH_INFO: `/v3/users/d96976d475824de28b978a80c8e095a0` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.483400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96976d475824de28b978a80c8e095a0)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:06.485380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:06.539879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96976d475824de28b978a80c8e095a0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.550075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96976d475824de28b978a80c8e095a0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.551425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96976d475824de28b978a80c8e095a0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.561377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:06.562195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96976d475824de28b978a80c8e095a0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.562947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:06.577365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681803d-6136-468c-89ad-3479faa58d1d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:06.582897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:06.583678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.583869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-19de2808-b28b-4b8e-aa05-e0e7dcfe7c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96976d475824de28b978a80c8e095a0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.584167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.584579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] PATH_INFO: `/v3/users/c770ab1dc73a4e5eb3176d33100f9084` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.585880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c770ab1dc73a4e5eb3176d33100f9084)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:06.589030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:06.599437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1102/2224] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:06 2022] DELETE /identity/v3/users/d96976d475824de28b978a80c8e095a0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:06.650471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.651142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.651313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.672473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c770ab1dc73a4e5eb3176d33100f9084'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.673202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c770ab1dc73a4e5eb3176d33100f9084'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.674175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c770ab1dc73a4e5eb3176d33100f9084'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.676728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:06.678498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:06.682780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c770ab1dc73a4e5eb3176d33100f9084'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.692716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63468df6-82bf-46eb-ab35-d523e154b79d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c770ab1dc73a4e5eb3176d33100f9084'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:06.715756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1123/2225] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:06 2022] DELETE /identity/v3/users/c770ab1dc73a4e5eb3176d33100f9084 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:06.719941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.720225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.720531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.739905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:06.741172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:06.748177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1103/2226] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:52:06.775596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:06.810620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:06.812238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:06.831005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1124/2227] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:52:06.837217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:06.839197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:06.841349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:06.893737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173b08b-5010-45c7-af41-6823e5deb937 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:06.897975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c905f-a093-4e7a-9a56-0a3d8138e6bc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:06.902198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:06.902926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.903203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.903486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.917113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:06.917939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:06.930224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1125/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:52:06.933010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:06.934464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:06.947248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622ceb-aa09-4225-aa0e-79810190b747 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:06.954391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:06.955100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.955450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.966196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] PATH_INFO: `/v3/projects/71d72c114dd545459024b05d1f5dcf62` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:06.968976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d72c114dd545459024b05d1f5dcf62)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:06.971580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:06.991516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe38d-4b34-4590-946c-3b555d704b01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:06.992133 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe38d-4b34-4590-946c-3b555d704b01 service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:06.992491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe38d-4b34-4590-946c-3b555d704b01 service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:06.992760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe38d-4b34-4590-946c-3b555d704b01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.038958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d72c114dd545459024b05d1f5dcf62', 'request_id': 'req-134002a8-8a5f-4e9c-8f89-2d2526303075'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.046877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d72c114dd545459024b05d1f5dcf62', 'request_id': 'req-134002a8-8a5f-4e9c-8f89-2d2526303075'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.053770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d72c114dd545459024b05d1f5dcf62', 'request_id': 'req-134002a8-8a5f-4e9c-8f89-2d2526303075'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.056470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfe38d-4b34-4590-946c-3b555d704b01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.058585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfe38d-4b34-4590-946c-3b555d704b01 service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.074050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1104/2229] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:52:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 12 10:52:07.075880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] The token cache is being invalidate because project 71d72c114dd545459024b05d1f5dcf62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:07.076977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.077676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-134002a8-8a5f-4e9c-8f89-2d2526303075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.079871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1126/2230] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:06 2022] DELETE /identity/v3/projects/71d72c114dd545459024b05d1f5dcf62 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:07.085716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8877c-dc6d-4b6a-b702-a0c7305208fc None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.086005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8877c-dc6d-4b6a-b702-a0c7305208fc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.086781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8877c-dc6d-4b6a-b702-a0c7305208fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.107119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d3c8877c-dc6d-4b6a-b702-a0c7305208fc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:07.108413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d3c8877c-dc6d-4b6a-b702-a0c7305208fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:07.243713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1105/2231] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:07.251185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061a395-8a0a-4914-b9da-3911b9d483fd None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.251623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061a395-8a0a-4914-b9da-3911b9d483fd None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.252018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061a395-8a0a-4914-b9da-3911b9d483fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.280161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a061a395-8a0a-4914-b9da-3911b9d483fd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:07.281666 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a061a395-8a0a-4914-b9da-3911b9d483fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:07.290929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1127/2232] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:07.296424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8877c-dc6d-4b6a-b702-a0c7305208fc None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.329483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8877c-dc6d-4b6a-b702-a0c7305208fc None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.340412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.341352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.341794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.342218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.343019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:52:07.343461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.345505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.348389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:07.366726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1106/2233] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:52:07.367882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061a395-8a0a-4914-b9da-3911b9d483fd None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.379445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061a395-8a0a-4914-b9da-3911b9d483fd None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.384233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.384887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.386071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.386313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.386970 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.389049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.408840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.411378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:07.433255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12bb7f-2a2d-49d5-8d43-108459958979 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.435569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1128/2234] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:52:07 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:52:07.458418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.504207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:07.530384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.531028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.531304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.531572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] PATH_INFO: `/v3/projects/d54ff82d86764c05b1b1bcf200df6741` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.533052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54ff82d86764c05b1b1bcf200df6741)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.536795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.566479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f419c0-3d50-460d-a65a-99cd5cfe2484 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.573960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.574894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.575182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.575711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.576369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.578601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.609391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1129/2235] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:52:07 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:07.618003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.622092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54ff82d86764c05b1b1bcf200df6741', 'request_id': 'req-c8d60bfe-0a17-4605-954c-b3b450d241c4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.629774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad6d22-958b-4e0d-8dd5-d9676b0ad1ff admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.642008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54ff82d86764c05b1b1bcf200df6741', 'request_id': 'req-c8d60bfe-0a17-4605-954c-b3b450d241c4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.655513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54ff82d86764c05b1b1bcf200df6741', 'request_id': 'req-c8d60bfe-0a17-4605-954c-b3b450d241c4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.677121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] The token cache is being invalidate because project d54ff82d86764c05b1b1bcf200df6741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:07.678330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.680369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.682225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1107/2236] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:07 2022] DELETE /identity/v3/projects/d54ff82d86764c05b1b1bcf200df6741 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:07.688570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.691434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:07.708861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.709485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.709732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.709995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.710831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.712471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.724022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1130/2237] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:07.727887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.729783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:07.743582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63806a9-3c45-4fbc-94d0-2b44eae2986f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.749582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.750387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.750639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.750968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.751837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.753539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.764790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1131/2238] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:07.769714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:07.783675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d60bfe-0a17-4605-954c-b3b450d241c4 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.787080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90bafa-d36b-4ab6-a322-2a94eb14a1c8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:07.788784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.789511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.789843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.790219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] PATH_INFO: `/v3/users/fa968ed5a86f4c088490a3965316afc8` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.791723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa968ed5a86f4c088490a3965316afc8)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.793753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.795608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c4577-f17b-46ef-bf7f-e80c5e165f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:07.797377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c4577-f17b-46ef-bf7f-e80c5e165f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.798031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c4577-f17b-46ef-bf7f-e80c5e165f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.798490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c4577-f17b-46ef-bf7f-e80c5e165f97 admin admin] PATH_INFO: `/v3/projects/8f392857545447fb9dcabdc13c1f2880/users/846a3cce3fae4d7a9a5fb4b9e497549b/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.819920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c4577-f17b-46ef-bf7f-e80c5e165f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f392857545447fb9dcabdc13c1f2880, user_id=846a3cce3fae4d7a9a5fb4b9e497549b, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:07.822759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c4577-f17b-46ef-bf7f-e80c5e165f97 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:07.829316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:07.855642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1132/2239] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:07 2022] PUT /identity/v3/projects/8f392857545447fb9dcabdc13c1f2880/users/846a3cce3fae4d7a9a5fb4b9e497549b/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:07.859629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:07.860240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:07.860752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:07.865554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa968ed5a86f4c088490a3965316afc8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.874928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa968ed5a86f4c088490a3965316afc8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.875697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa968ed5a86f4c088490a3965316afc8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.887517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa968ed5a86f4c088490a3965316afc8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.904928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa968ed5a86f4c088490a3965316afc8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:07.908591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] MFA Rules not processed for user `846a3cce3fae4d7a9a5fb4b9e497549b`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:07.910500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:07.915109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1108/2240] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:07 2022] DELETE /identity/v3/users/fa968ed5a86f4c088490a3965316afc8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:08.005422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1133/2241] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:08.019621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:08.021259 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:08.110863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6fe83-2776-49f9-b4e5-213b7fa8232f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:08.115957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:08.116604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:08.117158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:08.117158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:08.201852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:08.204161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:08.218610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1109/2242] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:08.264093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:08.265769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:08.273773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:08.408340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c0226-f70b-45a5-872a-074bcdd19113 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:08.542083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:08.543151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:08.543609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:08.544026 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] PATH_INFO: `/v3/projects/d7de6c6c59024d6582615f23d862e1c9` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:08.545568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7de6c6c59024d6582615f23d862e1c9)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:08.548412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:08.636059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7de6c6c59024d6582615f23d862e1c9', 'request_id': 'req-8247fdce-0cf9-4fef-8285-ae0fa50181e6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.644586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7de6c6c59024d6582615f23d862e1c9', 'request_id': 'req-8247fdce-0cf9-4fef-8285-ae0fa50181e6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.653092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7de6c6c59024d6582615f23d862e1c9', 'request_id': 'req-8247fdce-0cf9-4fef-8285-ae0fa50181e6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.680503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] The token cache is being invalidate because project d7de6c6c59024d6582615f23d862e1c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:08.681005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.681991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.684020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1134/2243] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:08 2022] DELETE /identity/v3/projects/d7de6c6c59024d6582615f23d862e1c9 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:08.688468 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:08.701035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f429f68-099d-465b-b9a7-b3b42f46ed4d service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:08.757733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:08.769371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:08.770649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:08.771223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:08.771792 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] PATH_INFO: `/v3/users/39f569069bbb42eeab4d6fc6977c10fa` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:08.774132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f569069bbb42eeab4d6fc6977c10fa)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:08.777341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:08.839058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f569069bbb42eeab4d6fc6977c10fa'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.857625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f569069bbb42eeab4d6fc6977c10fa'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.857625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f569069bbb42eeab4d6fc6977c10fa'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.860953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f569069bbb42eeab4d6fc6977c10fa'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.876584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7c54ee75-428f-4506-a18a-469982960ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f569069bbb42eeab4d6fc6977c10fa'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:08.886509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1110/2244] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:08 2022] DELETE /identity/v3/users/39f569069bbb42eeab4d6fc6977c10fa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:09.183459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:09.185203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:09.202262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247fdce-0cf9-4fef-8285-ae0fa50181e6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:09.254791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:09.275828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:09.276497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.276774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.277044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] PATH_INFO: `/v3/projects/2882aa531f3c4c539914592952dd5571` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.278314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2882aa531f3c4c539914592952dd5571)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:09.280223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:09.329070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2882aa531f3c4c539914592952dd5571', 'request_id': 'req-463631f1-4221-43d2-8279-9b6193d6e73d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:09.335382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2882aa531f3c4c539914592952dd5571', 'request_id': 'req-463631f1-4221-43d2-8279-9b6193d6e73d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:09.341364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2882aa531f3c4c539914592952dd5571', 'request_id': 'req-463631f1-4221-43d2-8279-9b6193d6e73d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:09.355799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] The token cache is being invalidate because project 2882aa531f3c4c539914592952dd5571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:09.356132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:09.356797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-463631f1-4221-43d2-8279-9b6193d6e73d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:09.358354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1135/2245] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:09 2022] DELETE /identity/v3/projects/2882aa531f3c4c539914592952dd5571 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:09.363328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafae49-9edb-4c77-a307-fcf69d40ffc7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.363610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafae49-9edb-4c77-a307-fcf69d40ffc7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.363877 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafae49-9edb-4c77-a307-fcf69d40ffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.384697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0cafae49-9edb-4c77-a307-fcf69d40ffc7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:09.386364 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0cafae49-9edb-4c77-a307-fcf69d40ffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:09.458486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1111/2246] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:52:09.481379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced9452-5c61-4c0d-8ea5-0a6928293179 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.481938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced9452-5c61-4c0d-8ea5-0a6928293179 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.482405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced9452-5c61-4c0d-8ea5-0a6928293179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.597989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0ced9452-5c61-4c0d-8ea5-0a6928293179 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:09.600931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0ced9452-5c61-4c0d-8ea5-0a6928293179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:09.634564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1136/2247] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:09.638868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafae49-9edb-4c77-a307-fcf69d40ffc7 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:09.654665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafae49-9edb-4c77-a307-fcf69d40ffc7 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:09.661316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:09.661934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.662216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.662476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.663145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:52:09.663426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:09.665262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:09.679205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1112/2248] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:52:09.683502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced9452-5c61-4c0d-8ea5-0a6928293179 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:09.693840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced9452-5c61-4c0d-8ea5-0a6928293179 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:09.700059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:09.700858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.701227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.701949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.702732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:09.704874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:09.752938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1137/2249] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:52:09 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:09.756871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:09.767833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e88937-a165-4718-b515-d0e9a8f27573 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:09.848289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:09.849012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.849284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.849551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.850253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:09.852214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:09.865812 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:09.886910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1113/2250] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:52:09 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:52:09.890501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:09.904901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de931bf0-edf8-49fe-bef9-2fa8bd8b3110 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:09.910924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:09.911771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.912104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.912483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.913188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:09.916681 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:09.932597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1138/2251] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:52:09.933637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:09.946184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9869b-a26d-4e1e-b1f6-f230c6019c4b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:09.951929 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2850bc6-c979-4e69-8bf0-e508af6db6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:09.953060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2850bc6-c979-4e69-8bf0-e508af6db6fd admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.953540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2850bc6-c979-4e69-8bf0-e508af6db6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:09.953989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2850bc6-c979-4e69-8bf0-e508af6db6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:09.954818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2850bc6-c979-4e69-8bf0-e508af6db6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:09.956850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2850bc6-c979-4e69-8bf0-e508af6db6fd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:09.974888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1114/2252] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:09.979764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:09.991010 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66cf8-08a4-4797-a07c-c697f7ff10d5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:09.998167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:09.999155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:09.999632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:10.000169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] PATH_INFO: `/v3/projects/ce501b5e17df4518a2ec76763bc3d81e/users/7bd68057dba244cabdb8efe86ee31b7e/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:10.025919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce501b5e17df4518a2ec76763bc3d81e, user_id=7bd68057dba244cabdb8efe86ee31b7e, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:10.030464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:10.052338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1139/2253] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:09 2022] PUT /identity/v3/projects/ce501b5e17df4518a2ec76763bc3d81e/users/7bd68057dba244cabdb8efe86ee31b7e/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:10.056030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5811b3-74d1-4422-9d7b-5c16eead1b4d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:10.056528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5811b3-74d1-4422-9d7b-5c16eead1b4d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:10.056999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5811b3-74d1-4422-9d7b-5c16eead1b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:10.078814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-be5811b3-74d1-4422-9d7b-5c16eead1b4d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:10.080226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-be5811b3-74d1-4422-9d7b-5c16eead1b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:10.169063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1115/2254] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:10.188860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:10.191040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:10.280419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba3029-c6d7-4d7f-8b7c-6ac25fc4aa1e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:10.300288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:10.301710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:10.302309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:10.303469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:10.323059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:10.325833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:10.343557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1140/2255] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:52:11.130238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d9d9-0c53-4ed4-84e4-78085d791783 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:11.131387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d9d9-0c53-4ed4-84e4-78085d791783 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:11.131915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d9d9-0c53-4ed4-84e4-78085d791783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:11.164367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-90a9d9d9-0c53-4ed4-84e4-78085d791783 None None] MFA Rules not processed for user `7bd68057dba244cabdb8efe86ee31b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:11.165752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-90a9d9d9-0c53-4ed4-84e4-78085d791783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:11.243820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1116/2256] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:11.259068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:11.261475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:11.273472 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:11.338205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bec11-297c-4f6e-822b-a7e42cc180a8 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:11.352483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:11.353119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:11.353465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:11.353971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:11.364962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:11.366454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:11.378890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1141/2257] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:12.981963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe31d2-7ef3-4b0f-807a-281976779f59 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:12.982534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe31d2-7ef3-4b0f-807a-281976779f59 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:12.982534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe31d2-7ef3-4b0f-807a-281976779f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:13.002370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-cebe31d2-7ef3-4b0f-807a-281976779f59 None None] MFA Rules not processed for user `7bd68057dba244cabdb8efe86ee31b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:13.003601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cebe31d2-7ef3-4b0f-807a-281976779f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:13.011090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1117/2258] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:19.755261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:19.756877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:19.775915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a476-16e8-4c8d-b04a-39c00ced00c4 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:19.781900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:19.782835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:19.783246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:19.784813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] PATH_INFO: `/v3/users/846a3cce3fae4d7a9a5fb4b9e497549b` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:19.786134 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=846a3cce3fae4d7a9a5fb4b9e497549b)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:19.806577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:19.930572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '846a3cce3fae4d7a9a5fb4b9e497549b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:19.932298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '846a3cce3fae4d7a9a5fb4b9e497549b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:19.933020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '846a3cce3fae4d7a9a5fb4b9e497549b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:19.946992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '846a3cce3fae4d7a9a5fb4b9e497549b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:19.970172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '846a3cce3fae4d7a9a5fb4b9e497549b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:20.091006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1142/2259] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:19 2022] DELETE /identity/v3/users/846a3cce3fae4d7a9a5fb4b9e497549b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:20.096460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:20.097504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:20.098114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:20.146151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:20.149198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:20.319098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1118/2260] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:20.344454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:20.349427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:20.464761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97d991-f41e-44fb-8365-be40810da376 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:20.475714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965befe-afcc-4851-ae08-9cae0f3cc2ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:20.476380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965befe-afcc-4851-ae08-9cae0f3cc2ac service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:20.476657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965befe-afcc-4851-ae08-9cae0f3cc2ac service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:20.476928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965befe-afcc-4851-ae08-9cae0f3cc2ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:20.503463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965befe-afcc-4851-ae08-9cae0f3cc2ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:20.505826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965befe-afcc-4851-ae08-9cae0f3cc2ac service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:20.520072 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1143/2261] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:20.828139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:20.829857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:20.844406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e6d5-b03a-4095-968f-842eb2360a69 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:20.849046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:20.849688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:20.849957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:20.850239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] PATH_INFO: `/v3/projects/8f392857545447fb9dcabdc13c1f2880` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:20.851408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f392857545447fb9dcabdc13c1f2880)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:20.853372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:20.854993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:20.904477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f392857545447fb9dcabdc13c1f2880', 'request_id': 'req-2536c8ca-4a7c-42ce-94cc-00a810996409'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:20.911816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f392857545447fb9dcabdc13c1f2880', 'request_id': 'req-2536c8ca-4a7c-42ce-94cc-00a810996409'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:20.918485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f392857545447fb9dcabdc13c1f2880', 'request_id': 'req-2536c8ca-4a7c-42ce-94cc-00a810996409'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:20.934090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] The token cache is being invalidate because project 8f392857545447fb9dcabdc13c1f2880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:20.941870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:20.942754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:20.956296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1119/2262] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:20 2022] DELETE /identity/v3/projects/8f392857545447fb9dcabdc13c1f2880 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:30.060343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6709d85-e3b5-44d9-b764-6864c65e1d3c None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:30.061574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6709d85-e3b5-44d9-b764-6864c65e1d3c None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:30.062067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6709d85-e3b5-44d9-b764-6864c65e1d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:30.102913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a6709d85-e3b5-44d9-b764-6864c65e1d3c None None] MFA Rules not processed for user `58e209063f5c454d87d9f743bedfc734`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:30.104901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a6709d85-e3b5-44d9-b764-6864c65e1d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:30.154251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:30.206992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1144/2263] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:30.753637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:30.755333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:30.832362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536c8ca-4a7c-42ce-94cc-00a810996409 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:30.837353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:30.837976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:30.838251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:30.838531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] PATH_INFO: `/v3/users/b4cea83d101e435d8d97df207761d053` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:30.840244 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4cea83d101e435d8d97df207761d053)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:30.843169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:30.896110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4cea83d101e435d8d97df207761d053'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:30.904513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4cea83d101e435d8d97df207761d053'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:30.905307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4cea83d101e435d8d97df207761d053'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:30.915118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4cea83d101e435d8d97df207761d053'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:30.928941 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4cea83d101e435d8d97df207761d053'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:30.936375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1120/2264] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:30 2022] DELETE /identity/v3/users/b4cea83d101e435d8d97df207761d053 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:30.942925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503d34-f548-423c-a8ad-5312e461f152 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:30.942925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503d34-f548-423c-a8ad-5312e461f152 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:30.942925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503d34-f548-423c-a8ad-5312e461f152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:30.967483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-9d503d34-f548-423c-a8ad-5312e461f152 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:30.968806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9d503d34-f548-423c-a8ad-5312e461f152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.032892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1145/2265] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:31.044644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:31.046023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.070202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:31.127308 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf0282-ec1b-4b46-92a2-e288ba363c7c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:31.133104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:31.133729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:31.134005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:31.134410 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:31.161421 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:31.163233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:31.179752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1121/2266] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:31.535080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:31.536661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:31.537114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:31.562244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:31.564024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.598725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:31.601525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.622945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136a281-f5ad-425d-9684-68a9290cee31 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:31.628749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:31.629509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:31.629850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:31.630289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] PATH_INFO: `/v3/projects/3161b29af3ee42f3826710e9948b1168` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:31.631563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3161b29af3ee42f3826710e9948b1168)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:31.633959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:31.651962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1146/2267] 10.222.0.23 () {58 vars in 967 bytes} [Fri Aug 12 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:31.669791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:31.671438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.703873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3161b29af3ee42f3826710e9948b1168', 'request_id': 'req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:31.712465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3161b29af3ee42f3826710e9948b1168', 'request_id': 'req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:31.722852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3161b29af3ee42f3826710e9948b1168', 'request_id': 'req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:31.749475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] The token cache is being invalidate because project 3161b29af3ee42f3826710e9948b1168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:31.750124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:31.750982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-eea1540f-8cce-4c7b-95ca-8ed0ec2bd564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:31.757098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1122/2268] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:31 2022] DELETE /identity/v3/projects/3161b29af3ee42f3826710e9948b1168 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:31.759462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba565cd-d928-4974-a71e-cb96fc4c929a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:31.759798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360c5c-6651-4dd8-8c0b-dcca84aab4cd None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:31.760396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360c5c-6651-4dd8-8c0b-dcca84aab4cd None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:31.760627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a360c5c-6651-4dd8-8c0b-dcca84aab4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:31.785721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:31.789965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-9a360c5c-6651-4dd8-8c0b-dcca84aab4cd None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:31.791558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9a360c5c-6651-4dd8-8c0b-dcca84aab4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.845448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:31.846238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:31.846601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:31.846944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:31.848920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.896944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1123/2269] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:52:31.898867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c88b5-e79e-45d0-bc57-acfb93cd4a74 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:31.899148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c88b5-e79e-45d0-bc57-acfb93cd4a74 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:31.899455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c88b5-e79e-45d0-bc57-acfb93cd4a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:31.953089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-846c88b5-e79e-45d0-bc57-acfb93cd4a74 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:31.953089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-846c88b5-e79e-45d0-bc57-acfb93cd4a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:31.964783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1124/2270] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:31.978636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c88b5-e79e-45d0-bc57-acfb93cd4a74 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:31.979204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:31.981101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:31.988628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c88b5-e79e-45d0-bc57-acfb93cd4a74 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:31.990955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1147/2271] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:52:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:52:31.994822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:31.995699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:31.996025 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:31.996422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:31.997051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:52:31.997439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:31.999424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.014598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1125/2272] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:52:32.018934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:32.030776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2f760-6db7-4cde-9cd1-b3cd1110ef3d service swift] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:32.035650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:32.036249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.036548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.036810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.037425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:32.039323 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.069262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1148/2273] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:52:32 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:32.073525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:32.086871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a1df-54af-4981-9c92-a1a7e6a67eeb admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:32.113664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:32.220562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:32.221440 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.221876 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.222247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.222899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:32.225030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.267681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1126/2274] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:52:32 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:32.269180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:32.280729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48566119-2f84-4492-b22f-cf4d64a1c538 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:32.311483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:32.312301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.312765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.313138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.313925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:32.316710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.351864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1149/2275] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:32.353930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:32.366076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e54e5-9f85-49f9-b575-bf4a8f8400d9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:32.371449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593141b1-9d14-4213-a745-1db7d3315e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:32.372062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593141b1-9d14-4213-a745-1db7d3315e89 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.372393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593141b1-9d14-4213-a745-1db7d3315e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.372742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593141b1-9d14-4213-a745-1db7d3315e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.373398 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593141b1-9d14-4213-a745-1db7d3315e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:32.375644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593141b1-9d14-4213-a745-1db7d3315e89 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.396500 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1127/2276] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:32.397668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:32.413997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311c551-34be-40ca-9327-678502c6c1a9 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:32.425050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:32.425740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.426022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.426304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] PATH_INFO: `/v3/projects/611782e0246346f685c51eca726c63ac/users/21bdb6abf6494e51962bf59fd7274635/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.445893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611782e0246346f685c51eca726c63ac, user_id=21bdb6abf6494e51962bf59fd7274635, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:32.448678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.467164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1150/2277] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:32 2022] PUT /identity/v3/projects/611782e0246346f685c51eca726c63ac/users/21bdb6abf6494e51962bf59fd7274635/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:32.477928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce276cd4-055a-4be0-9ff9-b5122cd64d00 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.478297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce276cd4-055a-4be0-9ff9-b5122cd64d00 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.478486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce276cd4-055a-4be0-9ff9-b5122cd64d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.511777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ce276cd4-055a-4be0-9ff9-b5122cd64d00 None None] MFA Rules not processed for user `21bdb6abf6494e51962bf59fd7274635`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:32.514108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ce276cd4-055a-4be0-9ff9-b5122cd64d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:32.613405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1128/2278] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:32.624654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:32.626974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:32.693283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36c61-a837-452d-bfa7-0018c1e19ab9 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:32.697611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c932abd-c98a-4974-bdcc-f266fe4863c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:32.698222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932abd-c98a-4974-bdcc-f266fe4863c3 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.698494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932abd-c98a-4974-bdcc-f266fe4863c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.698759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932abd-c98a-4974-bdcc-f266fe4863c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.709160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c932abd-c98a-4974-bdcc-f266fe4863c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:32.710687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c932abd-c98a-4974-bdcc-f266fe4863c3 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.723711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1151/2279] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:32.750292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce276cd4-055a-4be0-9ff9-b5122cd64d00 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:32.818807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce276cd4-055a-4be0-9ff9-b5122cd64d00 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:32.827228 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:32.828000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.828283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.828637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] PATH_INFO: `/v3/users/21bdb6abf6494e51962bf59fd7274635` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.829884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21bdb6abf6494e51962bf59fd7274635)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:32.831858 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:32.861956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:32.888004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21bdb6abf6494e51962bf59fd7274635'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:32.897267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21bdb6abf6494e51962bf59fd7274635'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:32.898055 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21bdb6abf6494e51962bf59fd7274635'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:32.907265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21bdb6abf6494e51962bf59fd7274635'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:32.921418 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21bdb6abf6494e51962bf59fd7274635'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:32.928834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1129/2280] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:32 2022] DELETE /identity/v3/users/21bdb6abf6494e51962bf59fd7274635 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:32.933309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:32.933636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:32.933969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:32.949047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:32.956373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:32.958587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:33.043242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1152/2281] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:33.053450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:33.054967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:33.123606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73abbac-fbbb-40af-8317-7245db583562 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:33.128949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ef32e-49a3-4cf5-be28-25e0f2581f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:33.129874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef32e-49a3-4cf5-be28-25e0f2581f05 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:33.130191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef32e-49a3-4cf5-be28-25e0f2581f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:33.130456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef32e-49a3-4cf5-be28-25e0f2581f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:33.140578 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ef32e-49a3-4cf5-be28-25e0f2581f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:33.142029 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ef32e-49a3-4cf5-be28-25e0f2581f05 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:33.156745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1130/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:33.729840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:33.737711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:33.750927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682675f-961d-40d3-a1de-581b68eccd2f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:33.755802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:33.756486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:33.756759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:33.757172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] PATH_INFO: `/v3/projects/611782e0246346f685c51eca726c63ac` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:33.758331 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=611782e0246346f685c51eca726c63ac)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:33.760288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:33.816638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611782e0246346f685c51eca726c63ac', 'request_id': 'req-9a897fc4-eac5-48ce-8d77-83d0cb623833'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:33.823703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611782e0246346f685c51eca726c63ac', 'request_id': 'req-9a897fc4-eac5-48ce-8d77-83d0cb623833'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:33.832144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611782e0246346f685c51eca726c63ac', 'request_id': 'req-9a897fc4-eac5-48ce-8d77-83d0cb623833'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:33.848197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] The token cache is being invalidate because project 611782e0246346f685c51eca726c63ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:33.848705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:33.849586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9a897fc4-eac5-48ce-8d77-83d0cb623833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:33.851299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1153/2283] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:33 2022] DELETE /identity/v3/projects/611782e0246346f685c51eca726c63ac => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:33.856831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41a182-f45b-4105-ba7f-c0284b53d576 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:33.857369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41a182-f45b-4105-ba7f-c0284b53d576 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:33.857618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41a182-f45b-4105-ba7f-c0284b53d576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:33.879023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-4c41a182-f45b-4105-ba7f-c0284b53d576 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:33.880568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4c41a182-f45b-4105-ba7f-c0284b53d576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:33.949225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1131/2284] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:33.958502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e512c-c4cc-493f-a36c-ddb7fbe9cd2a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:33.958857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e512c-c4cc-493f-a36c-ddb7fbe9cd2a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:33.959140 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e512c-c4cc-493f-a36c-ddb7fbe9cd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:33.980287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-888e512c-c4cc-493f-a36c-ddb7fbe9cd2a None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:33.981970 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-888e512c-c4cc-493f-a36c-ddb7fbe9cd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:33.993770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1154/2285] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:33.998394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41a182-f45b-4105-ba7f-c0284b53d576 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.009114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41a182-f45b-4105-ba7f-c0284b53d576 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.014696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.015955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.016289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.016581 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.017349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:52:34.017657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.019272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.035032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1132/2286] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:52:34.040986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e512c-c4cc-493f-a36c-ddb7fbe9cd2a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.055125 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e512c-c4cc-493f-a36c-ddb7fbe9cd2a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.060232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.060880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.061153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.061413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.062033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.073320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.096771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1155/2287] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:52:34 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:34.098213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.108815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809b8e-51fb-4eb9-95d4-7061618e18ba admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.142873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:34.179755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.180498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.180693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.180945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.181572 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.183653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.209129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1133/2288] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:52:34 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:52:34.213416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.230812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af66ae-ee5d-4d86-9461-6caa7f5f48d0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.236743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.238146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.238423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.238690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.239342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.240993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.264588 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1156/2289] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:34.265283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.281371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89f55e-8894-4e26-9470-edee63397c8c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.292650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9f39-3ad2-4378-8629-9045906c1f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.293635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9f39-3ad2-4378-8629-9045906c1f10 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.294019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9f39-3ad2-4378-8629-9045906c1f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.294358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9f39-3ad2-4378-8629-9045906c1f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.295255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9f39-3ad2-4378-8629-9045906c1f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.297324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9f39-3ad2-4378-8629-9045906c1f10 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.324329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1134/2290] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:34.326949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.348607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96cecb-e53f-4024-8ce0-491e8b6a3dc8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.357020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.358663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.359109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.359549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] PATH_INFO: `/v3/projects/6b39306332ef498bb63844d1f1a2532b/users/d9d947f239434197973138d2d7677126/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.380857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b39306332ef498bb63844d1f1a2532b, user_id=d9d947f239434197973138d2d7677126, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.385569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.411756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1157/2291] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:34 2022] PUT /identity/v3/projects/6b39306332ef498bb63844d1f1a2532b/users/d9d947f239434197973138d2d7677126/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:34.423931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.424341 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.425185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.463277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] MFA Rules not processed for user `d9d947f239434197973138d2d7677126`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:34.466064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:34.584131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1135/2292] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:34.595702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.597928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:34.646677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:34.648556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:34.663656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:34.698859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7056-6518-4f08-9eba-b369ec2be18a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.702602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.703210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.703514 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.703783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.712351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.713773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.723269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1158/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:34.736396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4f607-be7f-40a6-8d4d-df6516dc9afa None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:34.743069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:34.743707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:34.743997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:34.744352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] PATH_INFO: `/v3/users/d17fb4b6648e436ab20481059f81600a` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:34.745493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d17fb4b6648e436ab20481059f81600a)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:34.747493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:34.817189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:34.819808 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd17fb4b6648e436ab20481059f81600a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:34.831571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd17fb4b6648e436ab20481059f81600a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:34.832470 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd17fb4b6648e436ab20481059f81600a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:34.843621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd17fb4b6648e436ab20481059f81600a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:34.860698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-aa5bed4b-79f2-4a7d-92ce-b385af326eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd17fb4b6648e436ab20481059f81600a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:34.872636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1136/2294] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:52:34 2022] DELETE /identity/v3/users/d17fb4b6648e436ab20481059f81600a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:52:35.311363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:35.322961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e05081-a003-4751-bc04-c97da0c6df78 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:35.402277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:35.403087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:35.403414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:35.403745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] PATH_INFO: `/v3/projects/7795fcb0687b47aab64c34d161d24856` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:35.404952 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7795fcb0687b47aab64c34d161d24856)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:35.407058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:35.470361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7795fcb0687b47aab64c34d161d24856', 'request_id': 'req-832982fb-b5e9-4120-8f9f-4f90eb73c801'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:35.477179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7795fcb0687b47aab64c34d161d24856', 'request_id': 'req-832982fb-b5e9-4120-8f9f-4f90eb73c801'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:35.485080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7795fcb0687b47aab64c34d161d24856', 'request_id': 'req-832982fb-b5e9-4120-8f9f-4f90eb73c801'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:35.502395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] The token cache is being invalidate because project 7795fcb0687b47aab64c34d161d24856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:52:35.502842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:35.503335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-832982fb-b5e9-4120-8f9f-4f90eb73c801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:52:35.508329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1159/2295] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:52:35 2022] DELETE /identity/v3/projects/7795fcb0687b47aab64c34d161d24856 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:35.511123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4511-c4b0-4f37-94bc-7ffa19af6794 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:35.511409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4511-c4b0-4f37-94bc-7ffa19af6794 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:35.511683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4511-c4b0-4f37-94bc-7ffa19af6794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:35.543057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ae8b4511-c4b0-4f37-94bc-7ffa19af6794 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:35.544045 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ae8b4511-c4b0-4f37-94bc-7ffa19af6794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:35.618346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1137/2296] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:35.621002 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf9ee7-dc38-4734-bc0f-6d2204617a6d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:35.621535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf9ee7-dc38-4734-bc0f-6d2204617a6d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:35.621894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf9ee7-dc38-4734-bc0f-6d2204617a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:35.790339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-bebf9ee7-dc38-4734-bc0f-6d2204617a6d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:35.794395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-bebf9ee7-dc38-4734-bc0f-6d2204617a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:35.804301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1160/2297] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:35.807408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b4511-c4b0-4f37-94bc-7ffa19af6794 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:35.817493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b4511-c4b0-4f37-94bc-7ffa19af6794 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:35.822632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:35.823355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:35.823695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:35.824087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:35.824916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:52:35.825260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:35.827032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:35.841236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1138/2298] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:52:35.845424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf9ee7-dc38-4734-bc0f-6d2204617a6d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:35.859235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf9ee7-dc38-4734-bc0f-6d2204617a6d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:35.865099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:35.865904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:35.866044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:35.866266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:35.867770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:35.870046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:35.911615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:35.912984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1161/2299] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:52:35 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:35.923383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a240be9-23a5-4827-880c-360761cb05c9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:35.995667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:35.996754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:35.996754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:35.997001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:35.997608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:35.999535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:36.027004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1139/2300] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:52:35 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:52:36.028890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:36.040350 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf7319-d230-46ba-a5a1-5cdacd88ccdb admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:36.051634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:36.052254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:36.052552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:36.052822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:36.053688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:36.055430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:36.070503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1162/2301] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:52:36.072826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:36.089263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e592a3-ed62-44e1-8a98-f216ccca249d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:36.102687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1707e12-dbcc-4062-86da-780381125894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:36.103335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1707e12-dbcc-4062-86da-780381125894 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:36.103606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1707e12-dbcc-4062-86da-780381125894 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:36.103870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1707e12-dbcc-4062-86da-780381125894 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:36.104539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1707e12-dbcc-4062-86da-780381125894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:36.106654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1707e12-dbcc-4062-86da-780381125894 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:36.130128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1140/2302] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:52:36.134582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:36.145182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e09959-9036-4f06-91c5-4568b8255ce0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:36.150762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885163d4-49dc-4ac1-b106-75935f591f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:36.150995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885163d4-49dc-4ac1-b106-75935f591f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:36.151273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885163d4-49dc-4ac1-b106-75935f591f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:36.151538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885163d4-49dc-4ac1-b106-75935f591f0b admin admin] PATH_INFO: `/v3/projects/e2a00d05f2a94dc89c0f5d56a5ebfd49/users/abfdb6e149cf49faa0ddbcca20007391/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:36.181321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885163d4-49dc-4ac1-b106-75935f591f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2a00d05f2a94dc89c0f5d56a5ebfd49, user_id=abfdb6e149cf49faa0ddbcca20007391, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:36.185534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885163d4-49dc-4ac1-b106-75935f591f0b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:36.190430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:36.214105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:36.214597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:36.215796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:36.234023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1163/2303] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:52:36 2022] PUT /identity/v3/projects/e2a00d05f2a94dc89c0f5d56a5ebfd49/users/abfdb6e149cf49faa0ddbcca20007391/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:52:36.243745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] MFA Rules not processed for user `d9d947f239434197973138d2d7677126`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:36.243941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f144447-aeb2-4929-8c3a-8a9f60f6f30e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:36.244645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f144447-aeb2-4929-8c3a-8a9f60f6f30e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:36.245075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f144447-aeb2-4929-8c3a-8a9f60f6f30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:36.246084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:36.276363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:36.278719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8f144447-aeb2-4929-8c3a-8a9f60f6f30e None None] MFA Rules not processed for user `abfdb6e149cf49faa0ddbcca20007391`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:36.280001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8f144447-aeb2-4929-8c3a-8a9f60f6f30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:36.342394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1141/2304] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:36.372359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1164/2305] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:52:36.384295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:36.385713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:36.474161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3bffb-31db-405a-8857-593321c3d762 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:36.478567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca590f-c1d7-4f63-abf7-0eb0ac4ef7de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8571f3d80b0749729012353fcbbd9e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:36.479281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca590f-c1d7-4f63-abf7-0eb0ac4ef7de service glance] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:36.479613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca590f-c1d7-4f63-abf7-0eb0ac4ef7de service glance] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:36.479949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca590f-c1d7-4f63-abf7-0eb0ac4ef7de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:36.497159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca590f-c1d7-4f63-abf7-0eb0ac4ef7de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:36.497718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca590f-c1d7-4f63-abf7-0eb0ac4ef7de service glance] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:36.507851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1142/2306] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:52:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:52:37.131677 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:37.132046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:37.132453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:37.157232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] MFA Rules not processed for user `e0d5ff60d4b24fe0bbd771e8ed97db2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:37.158440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:37.232329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1165/2307] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:46.503756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=39 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:47.224133 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=59 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:52:56.843531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9938e1-468f-4cb5-a884-a590caacb0f4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:56.844026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9938e1-468f-4cb5-a884-a590caacb0f4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:56.844130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9938e1-468f-4cb5-a884-a590caacb0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:56.864938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-fd9938e1-468f-4cb5-a884-a590caacb0f4 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:52:56.866134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fd9938e1-468f-4cb5-a884-a590caacb0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:56.931490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1143/2308] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:52:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:52:56.944444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:52:56.945969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:52:57.025481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35442f9e-a41f-461f-9c5b-2636138177df None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:52:57.032465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:52:57.033079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:52:57.033351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:52:57.033623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:52:57.042675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:52:57.044288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:52:57.053168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1166/2309] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:52:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:06.924877 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=25 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:07.050284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=32 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:07.271150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dc778-7655-46d9-a6eb-2dcfd4efe92f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:07.271501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dc778-7655-46d9-a6eb-2dcfd4efe92f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:07.271831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dc778-7655-46d9-a6eb-2dcfd4efe92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:07.293514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-882dc778-7655-46d9-a6eb-2dcfd4efe92f None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:07.294776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-882dc778-7655-46d9-a6eb-2dcfd4efe92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:07.302460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1144/2310] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:53:07.325597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:07.338065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26ed43-b717-47d4-b16f-806d96b45fd9 service swift] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:07.344741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:07.345576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:07.345848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:07.346111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:07.355789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:07.357738 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:07.368177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1167/2311] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:07.905454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76caa-327b-4515-b1f8-7f8e5ddd42ba None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:07.905882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76caa-327b-4515-b1f8-7f8e5ddd42ba None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:07.906153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76caa-327b-4515-b1f8-7f8e5ddd42ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:07.925369 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-99b76caa-327b-4515-b1f8-7f8e5ddd42ba None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:07.926580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-99b76caa-327b-4515-b1f8-7f8e5ddd42ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:07.934388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1145/2312] 10.222.0.23 () {58 vars in 967 bytes} [Fri Aug 12 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:53:07.946734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:07.957597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19658-1d53-483c-b7e4-5f5807354cb7 service swift] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:07.962768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:07.963361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:07.963639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:07.963903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:07.972610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:07.974205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:07.982092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1168/2313] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:08.227516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9346ed-b7b9-41cb-9a4f-43a226220531 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:08.227821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9346ed-b7b9-41cb-9a4f-43a226220531 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:08.228090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9346ed-b7b9-41cb-9a4f-43a226220531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:08.248850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-eb9346ed-b7b9-41cb-9a4f-43a226220531 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:08.250081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-eb9346ed-b7b9-41cb-9a4f-43a226220531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:08.258822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1146/2314] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:53:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:08.276315 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:08.287017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c9f02-e0c4-4877-bc9e-a17d682ec2a9 service swift] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:08.292555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:08.293145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:08.293449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:08.293714 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:08.302139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:08.303783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:08.311954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1169/2315] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:53:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:08.653408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b0dda-9a63-43c7-8cb3-0ba2e1c911c4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:08.653676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b0dda-9a63-43c7-8cb3-0ba2e1c911c4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:08.653948 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b0dda-9a63-43c7-8cb3-0ba2e1c911c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:08.675483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a75b0dda-9a63-43c7-8cb3-0ba2e1c911c4 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:08.676735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a75b0dda-9a63-43c7-8cb3-0ba2e1c911c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:08.685704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1147/2316] 10.222.0.23 () {58 vars in 967 bytes} [Fri Aug 12 10:53:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:08.698118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:08.709355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533be35e-e01b-4e67-9c28-3221e76abc68 service swift] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:08.716096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261a814-5ee8-4ba7-9091-d5a5115e2dba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:08.716724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a814-5ee8-4ba7-9091-d5a5115e2dba service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:08.716996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a814-5ee8-4ba7-9091-d5a5115e2dba service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:08.717266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261a814-5ee8-4ba7-9091-d5a5115e2dba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:08.726198 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261a814-5ee8-4ba7-9091-d5a5115e2dba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:08.727778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261a814-5ee8-4ba7-9091-d5a5115e2dba service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:08.736313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1170/2317] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:53:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:15.169926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a75701-7654-4b84-b99c-70927f9f9bbc None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.170524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a75701-7654-4b84-b99c-70927f9f9bbc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.170796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a75701-7654-4b84-b99c-70927f9f9bbc None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.172433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1148/2318] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:53:15.178679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceab4ef-40a3-4277-b0ad-2f4902ad929d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.179170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceab4ef-40a3-4277-b0ad-2f4902ad929d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.179273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceab4ef-40a3-4277-b0ad-2f4902ad929d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.198798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0ceab4ef-40a3-4277-b0ad-2f4902ad929d None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:15.199987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0ceab4ef-40a3-4277-b0ad-2f4902ad929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:15.266993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1171/2319] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:15.279864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a75701-7654-4b84-b99c-70927f9f9bbc None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:15.282144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f4a75701-7654-4b84-b99c-70927f9f9bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:15.295465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a75701-7654-4b84-b99c-70927f9f9bbc None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:15.299721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac8ecd-f177-4838-9552-b8eab52ec63b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8571f3d80b0749729012353fcbbd9e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:15.300319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8ecd-f177-4838-9552-b8eab52ec63b service glance] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.300609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8ecd-f177-4838-9552-b8eab52ec63b service glance] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.300847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8ecd-f177-4838-9552-b8eab52ec63b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.313297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac8ecd-f177-4838-9552-b8eab52ec63b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:15.314758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac8ecd-f177-4838-9552-b8eab52ec63b service glance] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:15.325159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1149/2320] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:15.402986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d69db-f237-4647-8cee-e4f061573518 None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.403267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d69db-f237-4647-8cee-e4f061573518 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.403531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d69db-f237-4647-8cee-e4f061573518 None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.404980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1172/2321] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:53:15.409912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff765c-70f1-49f3-b69e-42589bad9696 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.410195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff765c-70f1-49f3-b69e-42589bad9696 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.410506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff765c-70f1-49f3-b69e-42589bad9696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.431120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-85ff765c-70f1-49f3-b69e-42589bad9696 None None] MFA Rules not processed for user `a8f3a56366d348bfb0a5c445905dc46b`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:15.432346 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-85ff765c-70f1-49f3-b69e-42589bad9696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:15.494082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1150/2322] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:53:15.506565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d69db-f237-4647-8cee-e4f061573518 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:15.508395 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d63d69db-f237-4647-8cee-e4f061573518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:15.521983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d69db-f237-4647-8cee-e4f061573518 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:15.525639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d004e17-6625-4ce2-9fa5-abd48fd1c9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:15.526223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004e17-6625-4ce2-9fa5-abd48fd1c9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.526500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004e17-6625-4ce2-9fa5-abd48fd1c9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.526760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d004e17-6625-4ce2-9fa5-abd48fd1c9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.533879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d004e17-6625-4ce2-9fa5-abd48fd1c9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:15.535311 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d004e17-6625-4ce2-9fa5-abd48fd1c9e1 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:15.542918 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1173/2323] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:15.646589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c83d-190a-4d5a-9e3a-dd12d88d02fc None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.646904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c83d-190a-4d5a-9e3a-dd12d88d02fc None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.647172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0c83d-190a-4d5a-9e3a-dd12d88d02fc None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.648553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1151/2324] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 10:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 10:53:15.656324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.656551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.656823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.677296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:15.678515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:15.688946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1174/2325] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:15.704031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c83d-190a-4d5a-9e3a-dd12d88d02fc None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:15.721704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0c83d-190a-4d5a-9e3a-dd12d88d02fc None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:15.727985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b1e24-ecc9-459a-a832-f5a367d90281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:15.728667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b1e24-ecc9-459a-a832-f5a367d90281 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:15.729038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b1e24-ecc9-459a-a832-f5a367d90281 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:15.729316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b1e24-ecc9-459a-a832-f5a367d90281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:15.741007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b1e24-ecc9-459a-a832-f5a367d90281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:15.742479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b1e24-ecc9-459a-a832-f5a367d90281 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:15.753807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1152/2326] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:18.291230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:18.293053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:18.362181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0f6d4-24e1-4653-afdc-dd455b7b2274 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:18.367152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:18.367781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:18.368056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:18.368357 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] PATH_INFO: `/v3/users/d9d947f239434197973138d2d7677126` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:18.369577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d947f239434197973138d2d7677126)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:18.371509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:18.394663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:18.423774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d947f239434197973138d2d7677126'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:18.424402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d947f239434197973138d2d7677126'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:18.425017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d947f239434197973138d2d7677126'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:18.431912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d947f239434197973138d2d7677126'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:18.440590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d947f239434197973138d2d7677126'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:18.459596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1175/2327] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:53:18 2022] DELETE /identity/v3/users/d9d947f239434197973138d2d7677126 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:18.464340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:18.464800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:18.464896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:18.487219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:18.489257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:18.552443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1153/2328] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:18.564046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:18.565546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:18.632822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb120b-5b4f-4a18-90ce-17e31af3d3e6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:18.636835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb7334-6dd1-4766-99f3-6ed64c8f0fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:18.637434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7334-6dd1-4766-99f3-6ed64c8f0fbc service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:18.637703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7334-6dd1-4766-99f3-6ed64c8f0fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:18.637965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7334-6dd1-4766-99f3-6ed64c8f0fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:18.648977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb7334-6dd1-4766-99f3-6ed64c8f0fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:18.650426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb7334-6dd1-4766-99f3-6ed64c8f0fbc service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:18.660884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1176/2329] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:18.883379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:18.884996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:18.898167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3c12d-e1b1-4290-9d5d-bf500f9c7088 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:18.916333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:18.917008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:18.917359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:18.917539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] PATH_INFO: `/v3/projects/6b39306332ef498bb63844d1f1a2532b` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:18.918685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b39306332ef498bb63844d1f1a2532b)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:18.921742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:18.983588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b39306332ef498bb63844d1f1a2532b', 'request_id': 'req-6af32c1d-505f-4509-858d-2f0c23e39f87'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:18.985571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:18.990558 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b39306332ef498bb63844d1f1a2532b', 'request_id': 'req-6af32c1d-505f-4509-858d-2f0c23e39f87'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:18.997143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b39306332ef498bb63844d1f1a2532b', 'request_id': 'req-6af32c1d-505f-4509-858d-2f0c23e39f87'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:19.010593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] The token cache is being invalidate because project 6b39306332ef498bb63844d1f1a2532b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:53:19.010856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:19.011464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6af32c1d-505f-4509-858d-2f0c23e39f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:19.013005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1154/2330] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:53:18 2022] DELETE /identity/v3/projects/6b39306332ef498bb63844d1f1a2532b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:19.018594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2ae43-ebf3-4dd4-adfb-8659ff2dcecf None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.018947 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2ae43-ebf3-4dd4-adfb-8659ff2dcecf None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.019313 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2ae43-ebf3-4dd4-adfb-8659ff2dcecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.040325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c9b2ae43-ebf3-4dd4-adfb-8659ff2dcecf None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:19.041521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c9b2ae43-ebf3-4dd4-adfb-8659ff2dcecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:19.101610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1177/2331] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:19.113887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7948d5d-9aa4-4aa9-99f9-57d9d96f3e45 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.114233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7948d5d-9aa4-4aa9-99f9-57d9d96f3e45 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.114525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7948d5d-9aa4-4aa9-99f9-57d9d96f3e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.138844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d7948d5d-9aa4-4aa9-99f9-57d9d96f3e45 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:19.140132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d7948d5d-9aa4-4aa9-99f9-57d9d96f3e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:19.148704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1155/2332] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:19.153245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2ae43-ebf3-4dd4-adfb-8659ff2dcecf None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:19.163321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2ae43-ebf3-4dd4-adfb-8659ff2dcecf None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:19.169099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:19.169688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.169951 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.170202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.170872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:53:19.171158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:19.172788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:19.184520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1178/2333] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:53:19.189171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7948d5d-9aa4-4aa9-99f9-57d9d96f3e45 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:19.198739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7948d5d-9aa4-4aa9-99f9-57d9d96f3e45 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:19.203811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:19.204469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.204747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.205016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.205637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:19.207529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:19.236814 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1156/2334] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:53:19 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:19.241693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:19.254077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ee5a1-a137-4c34-9563-d680c792b16e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:19.279798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:19.312096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:19.312830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.312943 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.313207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.313841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:19.315725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:19.338961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1179/2335] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:53:19 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:19.343292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:19.354680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f63ae-392f-4efd-ae9d-94897fa4c958 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:19.360452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:19.361170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.361484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.361823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.362511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:19.364223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:19.374576 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1157/2336] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:19.378877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:19.388912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a16b7-d56e-4cba-a7b6-770ba2c51f9e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:19.393595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b775a6-439a-44e1-bbb1-ff950493143d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:19.394179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b775a6-439a-44e1-bbb1-ff950493143d admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.394446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b775a6-439a-44e1-bbb1-ff950493143d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.394708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b775a6-439a-44e1-bbb1-ff950493143d admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.395642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b775a6-439a-44e1-bbb1-ff950493143d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:19.397237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b775a6-439a-44e1-bbb1-ff950493143d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:19.407071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1180/2337] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:19.411138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:19.421698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957b50-4184-47a0-9d6e-3cb02bfe7dd6 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:19.426540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:19.427201 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.427471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.427736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] PATH_INFO: `/v3/projects/807c809e71de43ebb4edecbab4bc4292/users/3f40eebcea60459ab0cf9233aea9b743/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.442320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] RBAC: Authorizing `identity:create_grant(project_id=807c809e71de43ebb4edecbab4bc4292, user_id=3f40eebcea60459ab0cf9233aea9b743, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:19.445018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:19.460114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1158/2338] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:19 2022] PUT /identity/v3/projects/807c809e71de43ebb4edecbab4bc4292/users/3f40eebcea60459ab0cf9233aea9b743/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:19.465305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698c5d-36b5-47a9-9606-636d9ee69739 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.465583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698c5d-36b5-47a9-9606-636d9ee69739 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.465842 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698c5d-36b5-47a9-9606-636d9ee69739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.485474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7d698c5d-36b5-47a9-9606-636d9ee69739 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:19.486650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7d698c5d-36b5-47a9-9606-636d9ee69739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:19.548590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1181/2339] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:19.563560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:19.565626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:19.626571 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f0297-b977-4c81-9ec1-ddf3287a857c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:19.631102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:19.631706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:19.631980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:19.632248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:19.642295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:19.643738 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:19.653528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1159/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:24.733792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee394b6-4260-42ae-9c0d-903362c9128a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:24.734254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee394b6-4260-42ae-9c0d-903362c9128a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:24.734353 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee394b6-4260-42ae-9c0d-903362c9128a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:24.779498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-eee394b6-4260-42ae-9c0d-903362c9128a None None] MFA Rules not processed for user `3f40eebcea60459ab0cf9233aea9b743`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:24.782118 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-eee394b6-4260-42ae-9c0d-903362c9128a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:24.882338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1182/2341] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:24.886869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:24.898689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d173-b206-4369-b0ff-6b7e0c7cb44a service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:24.908196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:24.909378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:24.909863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:24.910218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:24.911342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:24.914432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:24.957191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1160/2342] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:53:24 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:53:24.961367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee394b6-4260-42ae-9c0d-903362c9128a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:24.973024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee394b6-4260-42ae-9c0d-903362c9128a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:25.007741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:25.038056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:25.038765 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:25.039184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:25.039524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:25.040215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:25.042258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:25.069001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1183/2343] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:53:24 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:25.072968 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:25.083847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386e5b9-6663-4f43-b115-f0ba5c8817cb admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:25.088552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:25.089157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:25.089433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:25.089702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:25.090345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:25.091951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:25.106661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1161/2344] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:25.107913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:25.119209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273cf95-491d-4109-b529-529bec8a94c4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:25.124201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9f3f7-3d09-4022-a82a-7accd3abc9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:25.124891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9f3f7-3d09-4022-a82a-7accd3abc9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:25.125169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9f3f7-3d09-4022-a82a-7accd3abc9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:25.125434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9f3f7-3d09-4022-a82a-7accd3abc9e6 admin admin] PATH_INFO: `/v3/projects/9d829e430cf44f5882e1ccc2f7cb5c06/users/51b6307613b7496eabcf12273a4db967/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:25.143735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9f3f7-3d09-4022-a82a-7accd3abc9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d829e430cf44f5882e1ccc2f7cb5c06, user_id=51b6307613b7496eabcf12273a4db967, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:25.147122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9f3f7-3d09-4022-a82a-7accd3abc9e6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:25.165689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1184/2345] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:25 2022] PUT /identity/v3/projects/9d829e430cf44f5882e1ccc2f7cb5c06/users/51b6307613b7496eabcf12273a4db967/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:25.169631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:25.180347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009975b1-dd81-4b75-aa32-2d7143f88909 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:25.229481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:25.245691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:25.246368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:25.246833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:25.247281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:25.248191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:25.250291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:25.265347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1162/2346] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:25.270436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:25.270584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:25.270695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:25.270963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:25.285120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928abdf-8c82-489c-9262-7464c789bf93 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:25.291090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:25.291756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:25.292112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:25.292424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] PATH_INFO: `/v3/projects/9d829e430cf44f5882e1ccc2f7cb5c06/users/51b6307613b7496eabcf12273a4db967/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:25.294777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d829e430cf44f5882e1ccc2f7cb5c06, user_id=51b6307613b7496eabcf12273a4db967, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:25.298130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:25.299020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:25.300736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:25.313748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1163/2347] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:25 2022] PUT /identity/v3/projects/9d829e430cf44f5882e1ccc2f7cb5c06/users/51b6307613b7496eabcf12273a4db967/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:53:25.388042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1185/2348] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:53:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:25.402488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:25.404877 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:25.498803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588663f-1fd5-40ba-be7e-d7bfcd2ee630 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:25.510930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c0e6e-d3e3-41bc-8d20-5d2cab8093fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:25.511659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c0e6e-d3e3-41bc-8d20-5d2cab8093fd service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:25.512043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c0e6e-d3e3-41bc-8d20-5d2cab8093fd service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:25.512646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c0e6e-d3e3-41bc-8d20-5d2cab8093fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:25.600502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c0e6e-d3e3-41bc-8d20-5d2cab8093fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:25.609579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c0e6e-d3e3-41bc-8d20-5d2cab8093fd service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:25.621041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1164/2349] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:53:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:26.653070 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:26.655418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:26.709233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f10ace-9ee0-4ece-831d-f82f4dc694aa None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:26.879697 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:26.880529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:26.880795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:26.881059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] PATH_INFO: `/v3/users/abfdb6e149cf49faa0ddbcca20007391` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:26.882216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfdb6e149cf49faa0ddbcca20007391)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:26.886778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:27.022955 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfdb6e149cf49faa0ddbcca20007391'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:27.024184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfdb6e149cf49faa0ddbcca20007391'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:27.033040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfdb6e149cf49faa0ddbcca20007391'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:27.038034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:27.044682 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfdb6e149cf49faa0ddbcca20007391'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:27.071461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfdb6e149cf49faa0ddbcca20007391'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:27.131639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1186/2350] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:53:26 2022] DELETE /identity/v3/users/abfdb6e149cf49faa0ddbcca20007391 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:27.137235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:27.138438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:27.138798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:27.160751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:27.181424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:27.310484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1165/2351] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:27.327074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:27.327074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:27.473324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad56ffd-6acb-4d02-94c3-31f680f8aa18 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:27.485554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23854558-270e-44ec-b315-e3cf5b26baa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:27.486282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854558-270e-44ec-b315-e3cf5b26baa3 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:27.486622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854558-270e-44ec-b315-e3cf5b26baa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:27.486961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854558-270e-44ec-b315-e3cf5b26baa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:27.505627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854558-270e-44ec-b315-e3cf5b26baa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:27.507343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854558-270e-44ec-b315-e3cf5b26baa3 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:27.530848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1187/2352] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:28.328416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:28.330622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:28.355552 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e686a6c-12e0-4ab4-90c4-63254818d518 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:28.373230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:28.376007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:28.376554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:28.378287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] PATH_INFO: `/v3/projects/e2a00d05f2a94dc89c0f5d56a5ebfd49` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:28.383918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2a00d05f2a94dc89c0f5d56a5ebfd49)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:28.386394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:28.410616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:28.493423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2a00d05f2a94dc89c0f5d56a5ebfd49', 'request_id': 'req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:28.502434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2a00d05f2a94dc89c0f5d56a5ebfd49', 'request_id': 'req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:28.532204 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2a00d05f2a94dc89c0f5d56a5ebfd49', 'request_id': 'req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:28.555031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] The token cache is being invalidate because project e2a00d05f2a94dc89c0f5d56a5ebfd49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:53:28.555323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:28.556001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-28114f34-9a5d-4ca0-bf6a-d353b9aa2cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:28.560340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1166/2353] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:53:28 2022] DELETE /identity/v3/projects/e2a00d05f2a94dc89c0f5d56a5ebfd49 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:28.563297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b8909-d3f3-413b-a768-fc3e3044c6b7 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:28.563794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b8909-d3f3-413b-a768-fc3e3044c6b7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:28.563884 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b8909-d3f3-413b-a768-fc3e3044c6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:28.626538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c05b8909-d3f3-413b-a768-fc3e3044c6b7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:28.628060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c05b8909-d3f3-413b-a768-fc3e3044c6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:28.822921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1188/2354] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:28.833270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296d31d-642e-44d1-82c8-55acb159cab0 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:28.833647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296d31d-642e-44d1-82c8-55acb159cab0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:28.833995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296d31d-642e-44d1-82c8-55acb159cab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:28.877499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b296d31d-642e-44d1-82c8-55acb159cab0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:28.878943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b296d31d-642e-44d1-82c8-55acb159cab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:28.899677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1167/2355] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:28.904453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b8909-d3f3-413b-a768-fc3e3044c6b7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:28.915304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b8909-d3f3-413b-a768-fc3e3044c6b7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:28.941705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:28.944329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:28.944612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:28.945795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:28.946493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:53:28.946804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:28.948499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:29.018003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1189/2356] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:53:29.022707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296d31d-642e-44d1-82c8-55acb159cab0 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:29.035232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296d31d-642e-44d1-82c8-55acb159cab0 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:29.051833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:29.052488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:29.052879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:29.053018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:29.053662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:29.061717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:29.197786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1168/2357] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:53:29 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:29.200604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:29.211822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb73e5d-db02-453b-bcf5-51bcc0c17c9d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:29.379631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:29.390724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:29.391605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:29.392021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:29.392489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:29.393290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:29.396402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:29.461555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1190/2358] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:53:29 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:29.465593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:29.485164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94690cae-1731-4bb3-b4e5-0c099fe7b4d0 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:29.529305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:29.532951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:29.533251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:29.533520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:29.538592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:29.540291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:29.568827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1169/2359] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:29.573008 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:29.591266 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a302289-984b-4236-b899-8ab602e63a81 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:29.605565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569e157-029a-470c-a3af-c2fc5faea956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:29.609017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569e157-029a-470c-a3af-c2fc5faea956 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:29.609397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569e157-029a-470c-a3af-c2fc5faea956 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:29.609961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569e157-029a-470c-a3af-c2fc5faea956 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:29.611946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569e157-029a-470c-a3af-c2fc5faea956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:29.615249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569e157-029a-470c-a3af-c2fc5faea956 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:29.643935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1191/2360] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:53:29.648396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:29.660493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c3274-2a4b-4e39-9a67-908ebc4cac10 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:29.670138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:29.671680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:29.671962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:29.672237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] PATH_INFO: `/v3/projects/27bac523646144ec82b7bb3fa4c753c2/users/d438607f074043caaaa382ec7d8355af/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:29.692984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bac523646144ec82b7bb3fa4c753c2, user_id=d438607f074043caaaa382ec7d8355af, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:29.696677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:29.725407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1170/2361] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:29 2022] PUT /identity/v3/projects/27bac523646144ec82b7bb3fa4c753c2/users/d438607f074043caaaa382ec7d8355af/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:29.726448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867a2a-3e2d-464d-920c-d40ab14f7671 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:29.726727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867a2a-3e2d-464d-920c-d40ab14f7671 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:29.726988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867a2a-3e2d-464d-920c-d40ab14f7671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:29.777801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7e867a2a-3e2d-464d-920c-d40ab14f7671 None None] MFA Rules not processed for user `d438607f074043caaaa382ec7d8355af`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:29.779112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7e867a2a-3e2d-464d-920c-d40ab14f7671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:29.883868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1192/2362] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:29.896491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:29.898461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:30.024403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b428a-dbf6-4e17-b1d3-9d34df815afd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:30.035925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:30.036762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:30.037140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:30.037510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:30.057563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:30.060357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:30.102489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1171/2363] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:33.268352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:33.269416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:33.269848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:33.320326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] MFA Rules not processed for user `51b6307613b7496eabcf12273a4db967`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:33.321723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:33.345261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:33.431072 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:33.494036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1193/2364] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:33.495120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d2af-b69b-4077-8601-1a39c46b37a8 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:33.507889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:33.510676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:33.512883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:33.513655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:33.517080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:33.517723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] PATH_INFO: `/v3/users/d438607f074043caaaa382ec7d8355af` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:33.520881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d438607f074043caaaa382ec7d8355af)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:33.526882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:33.647290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21460b1b-245a-41b0-918b-0eb964f94718 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:33.656722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8571f3d80b0749729012353fcbbd9e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:33.657524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:33.657900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:33.658248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:33.661636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:33.677194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:33.677424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd438607f074043caaaa382ec7d8355af'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:33.690062 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd438607f074043caaaa382ec7d8355af'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:33.697049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd438607f074043caaaa382ec7d8355af'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:33.723454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd438607f074043caaaa382ec7d8355af'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:33.745172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-01f96182-2a35-4690-99b8-15ce84b28970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd438607f074043caaaa382ec7d8355af'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:33.758386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1172/2365] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:53:33 2022] DELETE /identity/v3/users/d438607f074043caaaa382ec7d8355af => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:33.763358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c5943-35ea-426c-bb58-c4d7937e3ad0 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:33.763679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c5943-35ea-426c-bb58-c4d7937e3ad0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:33.764755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c5943-35ea-426c-bb58-c4d7937e3ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:33.789099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-334c5943-35ea-426c-bb58-c4d7937e3ad0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:33.790332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-334c5943-35ea-426c-bb58-c4d7937e3ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:33.811945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:33.813614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:33.825796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1194/2366] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:53:33.927021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1173/2367] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:53:33.939364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:33.946154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:34.062531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472a054-9de4-46b1-b22e-779b494278ea service glance] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:34.096416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:34.096936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:34.097211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:34.136337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:34.137600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:34.159770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:34.160764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:34.161156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:34.161611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:34.176281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:34.182871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:34.194537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1195/2368] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:34.232862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:34.249563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1174/2369] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:53:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:53:34.281112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:34.282862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:34.411834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af2180-43b0-4c3e-9bbe-3a06837d63f6 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:34.419758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62812de-3e1f-4003-9fb5-dc77d36e9c44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:34.425298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62812de-3e1f-4003-9fb5-dc77d36e9c44 service swift] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:34.425656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62812de-3e1f-4003-9fb5-dc77d36e9c44 service swift] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:34.425935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62812de-3e1f-4003-9fb5-dc77d36e9c44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:34.439863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=100 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:34.445207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62812de-3e1f-4003-9fb5-dc77d36e9c44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:34.448450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62812de-3e1f-4003-9fb5-dc77d36e9c44 service swift] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:34.474933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1196/2370] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:53:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:34.832370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:34.833959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:34.859783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c37383-2a56-41f0-b3fb-a2b3794643f4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:34.877010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:34.877756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:34.878091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:34.878901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] PATH_INFO: `/v3/projects/27bac523646144ec82b7bb3fa4c753c2` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:34.881990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bac523646144ec82b7bb3fa4c753c2)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:34.886331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:35.042370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bac523646144ec82b7bb3fa4c753c2', 'request_id': 'req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:35.048890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bac523646144ec82b7bb3fa4c753c2', 'request_id': 'req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:35.063827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bac523646144ec82b7bb3fa4c753c2', 'request_id': 'req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:35.090160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] The token cache is being invalidate because project 27bac523646144ec82b7bb3fa4c753c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:53:35.090471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:35.091608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-292ba4ab-e419-449a-a9a6-4ee3b0a68b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:35.098445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1175/2371] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:53:34 2022] DELETE /identity/v3/projects/27bac523646144ec82b7bb3fa4c753c2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:35.100217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37cf48-fc73-4fc5-a668-9e1f4d4be3b8 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:35.100963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37cf48-fc73-4fc5-a668-9e1f4d4be3b8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:35.101358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37cf48-fc73-4fc5-a668-9e1f4d4be3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:35.167416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-4e37cf48-fc73-4fc5-a668-9e1f4d4be3b8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:35.171308 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4e37cf48-fc73-4fc5-a668-9e1f4d4be3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:35.363130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1197/2372] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:35.369902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cb2a8-6d77-457c-9a6c-99f2e930730f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:35.370209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cb2a8-6d77-457c-9a6c-99f2e930730f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:35.370449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cb2a8-6d77-457c-9a6c-99f2e930730f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:35.404946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a93cb2a8-6d77-457c-9a6c-99f2e930730f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:35.406675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a93cb2a8-6d77-457c-9a6c-99f2e930730f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:35.428879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1176/2373] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:35.433363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37cf48-fc73-4fc5-a668-9e1f4d4be3b8 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:35.448465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37cf48-fc73-4fc5-a668-9e1f4d4be3b8 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:35.461578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:35.462209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:35.462481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:35.462747 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:35.463591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:53:35.464004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:35.465932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:35.484488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1198/2374] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:53:35.488912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cb2a8-6d77-457c-9a6c-99f2e930730f None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:35.513073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cb2a8-6d77-457c-9a6c-99f2e930730f None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:35.531679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:35.532432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:35.532770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:35.533130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:35.533837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:35.538456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:35.617715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1177/2375] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:53:35 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:35.621995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:35.639646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94438ed0-1676-4a41-aec5-b17767b12a63 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:35.835454 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:35.836419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:35.836843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:35.837284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:35.840225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:35.843703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:35.900899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1199/2376] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:53:35 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:35.905969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:35.923088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44393143-0a03-47ad-85ad-c1559b44849c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:35.931938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:35.932165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:35.932559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:35.932686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:35.933983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:35.951772 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:35.979363 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1178/2377] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:35.984612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:36.011852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bbf82-f523-476b-b5c3-d4d4951170c8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:36.018248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9c067-d4e3-48a9-910b-f55e4a84da68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:36.019232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9c067-d4e3-48a9-910b-f55e4a84da68 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:36.019662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9c067-d4e3-48a9-910b-f55e4a84da68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:36.020007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9c067-d4e3-48a9-910b-f55e4a84da68 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:36.021095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9c067-d4e3-48a9-910b-f55e4a84da68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:36.023219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9c067-d4e3-48a9-910b-f55e4a84da68 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:36.043944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1200/2378] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:36.049922 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:36.070907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba0625-ae3d-4e83-a164-606c8c6d11c8 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:36.076286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:36.077293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:36.077471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:36.077835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] PATH_INFO: `/v3/projects/ec60ad2f28c447f093c5cba7c63042bf/users/cebba78114d44808bb83e53c15203d2f/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:36.113176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec60ad2f28c447f093c5cba7c63042bf, user_id=cebba78114d44808bb83e53c15203d2f, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:36.116894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:36.151256 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1179/2379] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:36 2022] PUT /identity/v3/projects/ec60ad2f28c447f093c5cba7c63042bf/users/cebba78114d44808bb83e53c15203d2f/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:36.166078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:36.166700 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:36.167317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:36.229035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] MFA Rules not processed for user `cebba78114d44808bb83e53c15203d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:36.231018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:36.437318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:36.476666 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1201/2380] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:36.487866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:36.489957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:36.629382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afec2e-6682-4500-8ae6-eaeacf21c5c6 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:36.661136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:36.661763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:36.662037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:36.662297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:36.683457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:36.688030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:36.724675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1180/2381] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:42.242201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:42.243836 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:42.359258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711be9b-ae6c-44dc-be09-0ff50a53e904 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:42.364919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af917d-14f9-4688-a036-1124a5ab71fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:42.365589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af917d-14f9-4688-a036-1124a5ab71fc service placement] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:42.365971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af917d-14f9-4688-a036-1124a5ab71fc service placement] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:42.366120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af917d-14f9-4688-a036-1124a5ab71fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:42.368148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-99af917d-14f9-4688-a036-1124a5ab71fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:42.390863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af917d-14f9-4688-a036-1124a5ab71fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:42.392570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af917d-14f9-4688-a036-1124a5ab71fc service placement] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:42.405442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1202/2382] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:44.614238 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:44.636557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:44.688141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde30e10-2a72-4877-b8a1-79c7b36d0417 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:44.692775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:44.693407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:44.693682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:44.693953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] PATH_INFO: `/v3/users/cebba78114d44808bb83e53c15203d2f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:44.695083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cebba78114d44808bb83e53c15203d2f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:44.697068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:44.753164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cebba78114d44808bb83e53c15203d2f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:44.760795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cebba78114d44808bb83e53c15203d2f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:44.761553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cebba78114d44808bb83e53c15203d2f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:44.769314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cebba78114d44808bb83e53c15203d2f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:44.787028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cebba78114d44808bb83e53c15203d2f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:44.793643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1181/2383] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:53:44 2022] DELETE /identity/v3/users/cebba78114d44808bb83e53c15203d2f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:44.798669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:44.798669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:44.799142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:44.819382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:44.820707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:44.885336 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1203/2384] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:44.896327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:44.897875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:44.956270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996ca2-a952-4a5e-8070-14f2a9891a81 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:44.959944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddc9ea-f12c-4f8d-8b31-b50197d58532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:44.960577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc9ea-f12c-4f8d-8b31-b50197d58532 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:44.960851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc9ea-f12c-4f8d-8b31-b50197d58532 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:44.961117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddc9ea-f12c-4f8d-8b31-b50197d58532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:44.969861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddc9ea-f12c-4f8d-8b31-b50197d58532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:44.971296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddc9ea-f12c-4f8d-8b31-b50197d58532 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:44.980723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1182/2385] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:45.278903 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.280397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:45.292687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8bec5-be5f-4335-baf2-f09cf6f5ac24 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:45.297498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:45.298163 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.298442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.298705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] PATH_INFO: `/v3/projects/ec60ad2f28c447f093c5cba7c63042bf` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.299831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec60ad2f28c447f093c5cba7c63042bf)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:45.301788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:45.352021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec60ad2f28c447f093c5cba7c63042bf', 'request_id': 'req-fdc81cf0-c827-4615-8b16-c23fc300ac4c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:45.358196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec60ad2f28c447f093c5cba7c63042bf', 'request_id': 'req-fdc81cf0-c827-4615-8b16-c23fc300ac4c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:45.363612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec60ad2f28c447f093c5cba7c63042bf', 'request_id': 'req-fdc81cf0-c827-4615-8b16-c23fc300ac4c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:45.377881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] The token cache is being invalidate because project ec60ad2f28c447f093c5cba7c63042bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:53:45.378161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:45.378784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fdc81cf0-c827-4615-8b16-c23fc300ac4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:53:45.380282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1204/2386] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:53:45 2022] DELETE /identity/v3/projects/ec60ad2f28c447f093c5cba7c63042bf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:45.385632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298719-8ff8-40fd-8199-fa6673ad0550 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.385914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298719-8ff8-40fd-8199-fa6673ad0550 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.386243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298719-8ff8-40fd-8199-fa6673ad0550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.407649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-68298719-8ff8-40fd-8199-fa6673ad0550 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:45.408971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-68298719-8ff8-40fd-8199-fa6673ad0550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:45.447405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:45.469521 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1183/2387] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:45.477559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97805493-2c2d-47bc-ba3d-b55ba35ffd97 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.477829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97805493-2c2d-47bc-ba3d-b55ba35ffd97 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.478088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97805493-2c2d-47bc-ba3d-b55ba35ffd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.486023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:45.500065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-97805493-2c2d-47bc-ba3d-b55ba35ffd97 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:45.501301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-97805493-2c2d-47bc-ba3d-b55ba35ffd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:45.509486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1205/2388] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:45.513766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298719-8ff8-40fd-8199-fa6673ad0550 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.524021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298719-8ff8-40fd-8199-fa6673ad0550 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:45.529164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:45.529951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.530415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.530687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.531319 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:53:45.531613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:45.533261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:45.544716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1184/2389] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:53:45.549170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97805493-2c2d-47bc-ba3d-b55ba35ffd97 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.559172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97805493-2c2d-47bc-ba3d-b55ba35ffd97 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:45.565683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:45.566286 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.566568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.566817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.567442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:45.569543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:45.599750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1206/2390] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:53:45 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:45.604044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.614870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa5315-46f7-451d-9168-d40851427de6 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:45.673231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:45.673831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.674217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.674492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.675163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:45.677123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:45.701220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1185/2391] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:53:45 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:45.707255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.722808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0133887-cb0e-4783-9c66-2a2401adb7ce admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:45.730906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:45.731796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.732165 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.732585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.733450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:45.735099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:45.745880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1207/2392] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:45.750309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.761401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468d6c1-3437-4412-bdf5-7c332d8510e1 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:45.767273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bd9b0-0d73-4fdf-a9b8-101a512e533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:45.767870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd9b0-0d73-4fdf-a9b8-101a512e533c admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.768140 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd9b0-0d73-4fdf-a9b8-101a512e533c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.768430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd9b0-0d73-4fdf-a9b8-101a512e533c admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.769115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd9b0-0d73-4fdf-a9b8-101a512e533c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:45.771075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd9b0-0d73-4fdf-a9b8-101a512e533c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:45.782692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1186/2393] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:45.786965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.799213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ec8c8-b5aa-4905-977d-1d96496068f6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:45.807699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf0db4-affc-4c90-9057-04566be7b579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:45.808679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf0db4-affc-4c90-9057-04566be7b579 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.809004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf0db4-affc-4c90-9057-04566be7b579 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.809349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf0db4-affc-4c90-9057-04566be7b579 admin admin] PATH_INFO: `/v3/projects/89943304402242278f5614ea5bde6092/users/922f49fa7d324ebdade043e1be1a476f/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.832477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf0db4-affc-4c90-9057-04566be7b579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89943304402242278f5614ea5bde6092, user_id=922f49fa7d324ebdade043e1be1a476f, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:45.835139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf0db4-affc-4c90-9057-04566be7b579 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:45.850821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1208/2394] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:45 2022] PUT /identity/v3/projects/89943304402242278f5614ea5bde6092/users/922f49fa7d324ebdade043e1be1a476f/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:53:45.856534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.856952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.857220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.878236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:45.879427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:45.938640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404de04-6eb0-4b2f-ab58-08a0f6b092d1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:45.939062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404de04-6eb0-4b2f-ab58-08a0f6b092d1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:45.939447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404de04-6eb0-4b2f-ab58-08a0f6b092d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:45.946702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1187/2395] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:45.956785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:45.958444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:45.967761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2404de04-6eb0-4b2f-ab58-08a0f6b092d1 None None] MFA Rules not processed for user `3f40eebcea60459ab0cf9233aea9b743`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:45.969664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2404de04-6eb0-4b2f-ab58-08a0f6b092d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:46.032224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:46.036582 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b8c0f-c09e-4ab2-ae18-b5a76b4bcb36 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:46.041077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:46.041991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:46.042373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:46.042520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1209/2396] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:46.042621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:46.052435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:46.053891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:46.063589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1188/2397] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:53:48.892963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763dbe-46dd-452b-86fe-d0c9d1122cff None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:48.894911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763dbe-46dd-452b-86fe-d0c9d1122cff None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:48.897035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763dbe-46dd-452b-86fe-d0c9d1122cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:48.944605 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ae763dbe-46dd-452b-86fe-d0c9d1122cff None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:48.946886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ae763dbe-46dd-452b-86fe-d0c9d1122cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:49.065984 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1210/2398] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:53:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:49.072784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:49.074177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:49.153759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427023c2-d4e6-49a1-8e22-fe20528f2bc0 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:49.161976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666a9c7-49c4-4c46-a250-b16b8c135fc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:49.162603 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a9c7-49c4-4c46-a250-b16b8c135fc3 service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:49.162879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a9c7-49c4-4c46-a250-b16b8c135fc3 service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:49.163146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a9c7-49c4-4c46-a250-b16b8c135fc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:49.173087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666a9c7-49c4-4c46-a250-b16b8c135fc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:49.174720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666a9c7-49c4-4c46-a250-b16b8c135fc3 service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:49.199103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1189/2399] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:53:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:53:52.027212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7e759-52ec-4532-a748-31cb0c29bde1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:52.028384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7e759-52ec-4532-a748-31cb0c29bde1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:52.028852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7e759-52ec-4532-a748-31cb0c29bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:52.057548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-60b7e759-52ec-4532-a748-31cb0c29bde1 None None] MFA Rules not processed for user `7bd68057dba244cabdb8efe86ee31b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:52.059715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-60b7e759-52ec-4532-a748-31cb0c29bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:52.113990 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6623f3-b172-4e8f-8781-58378f025b62 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:52.114748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6623f3-b172-4e8f-8781-58378f025b62 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:52.114748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6623f3-b172-4e8f-8781-58378f025b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:52.161465 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:52.189300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7e6623f3-b172-4e8f-8781-58378f025b62 None None] MFA Rules not processed for user `922f49fa7d324ebdade043e1be1a476f`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:53:52.190538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7e6623f3-b172-4e8f-8781-58378f025b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:53:52.205423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1211/2400] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:52.374290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1190/2401] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:53:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:53:52.378422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7e759-52ec-4532-a748-31cb0c29bde1 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:52.402275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7e759-52ec-4532-a748-31cb0c29bde1 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:52.415438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:52.416203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:52.416372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:52.416645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:52.417295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:52.419215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:52.469968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1212/2402] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:53:52 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:52.475866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6623f3-b172-4e8f-8781-58378f025b62 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:52.505640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6623f3-b172-4e8f-8781-58378f025b62 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:52.680721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:52.682699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:52.682963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:52.683657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:52.684486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:52.687504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:52.728863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:53:52.741049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1191/2403] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:53:52 2022] POST /identity/v3/users => generated 296 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:53:52.743922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:52.758306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e016-8845-48ab-99dd-19453b84b33c admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:52.774394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:52.775185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:52.775557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:52.776000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:52.776936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:52.778841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:52.817879 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:52.819760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1213/2404] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:52.836226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35275c2-d2b4-4146-8c47-31728eb1a6f2 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:52.842334 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:52.843727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:52.844324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:52.844762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] PATH_INFO: `/v3/projects/a61f3df50a194d01bc1cf39a9963ba2a/users/683fe550384b4a31ba0e795fda8818ed/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:52.868176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61f3df50a194d01bc1cf39a9963ba2a, user_id=683fe550384b4a31ba0e795fda8818ed, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:52.871333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:52.899132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:52.899524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1192/2405] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:52 2022] PUT /identity/v3/projects/a61f3df50a194d01bc1cf39a9963ba2a/users/683fe550384b4a31ba0e795fda8818ed/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:53:52.912405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a859f1-d95d-4169-a788-e6a80bed6f45 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:53.031530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e04de-5622-4fe8-9f13-706665a35515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:53.033369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e04de-5622-4fe8-9f13-706665a35515 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:53.033897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e04de-5622-4fe8-9f13-706665a35515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:53.034314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e04de-5622-4fe8-9f13-706665a35515 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:53.035428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e04de-5622-4fe8-9f13-706665a35515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:53.038839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e04de-5622-4fe8-9f13-706665a35515 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:53.051651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1214/2406] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:53:53.055732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:53:53.074307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4f450-a211-4c3a-9907-acafb2b488de admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:53:53.083661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:53:53.084513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:53:53.084793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:53:53.085065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] PATH_INFO: `/v3/projects/a61f3df50a194d01bc1cf39a9963ba2a/users/683fe550384b4a31ba0e795fda8818ed/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:53:53.087779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61f3df50a194d01bc1cf39a9963ba2a, user_id=683fe550384b4a31ba0e795fda8818ed, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:53:53.090316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:53:53.108105 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1193/2407] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:53:53 2022] PUT /identity/v3/projects/a61f3df50a194d01bc1cf39a9963ba2a/users/683fe550384b4a31ba0e795fda8818ed/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:54:01.610185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826695-5b5b-4b3c-ba81-40632a728c38 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:01.610185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826695-5b5b-4b3c-ba81-40632a728c38 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:01.611083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826695-5b5b-4b3c-ba81-40632a728c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:01.679491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-63826695-5b5b-4b3c-ba81-40632a728c38 None None] MFA Rules not processed for user `683fe550384b4a31ba0e795fda8818ed`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:54:01.681058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-63826695-5b5b-4b3c-ba81-40632a728c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:01.824158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1215/2408] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:54:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:54:01.831957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:54:01.834159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:01.946220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23c4a1-203b-4304-8a57-57926232c341 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:54:01.952020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:54:01.952666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:01.952938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:01.953222 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:02.032040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:54:02.033661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:54:02.044061 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1194/2409] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:54:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:54:03.416152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c344a1-8da9-44a6-8655-375c29376f10 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:03.417481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c344a1-8da9-44a6-8655-375c29376f10 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:03.417834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c344a1-8da9-44a6-8655-375c29376f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:03.440637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-18c344a1-8da9-44a6-8655-375c29376f10 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:54:03.442067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-18c344a1-8da9-44a6-8655-375c29376f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:03.501004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:54:03.536398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1216/2410] 10.222.0.23 () {58 vars in 967 bytes} [Fri Aug 12 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:54:03.537628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:54:03.613733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:54:03.624482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f00fd3-170c-4847-9cda-7c952e7141fe service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:54:03.631219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285e930-98fe-4031-9ccd-f8f2c74b2acd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:54:03.632017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285e930-98fe-4031-9ccd-f8f2c74b2acd service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:03.632512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285e930-98fe-4031-9ccd-f8f2c74b2acd service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:03.632935 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285e930-98fe-4031-9ccd-f8f2c74b2acd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:03.645247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285e930-98fe-4031-9ccd-f8f2c74b2acd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:54:03.648696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285e930-98fe-4031-9ccd-f8f2c74b2acd service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:54:03.663519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1195/2411] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:54:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:54:13.521466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=6 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:54:13.656195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:54:20.366633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:20.367132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:20.367293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:20.387906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] MFA Rules not processed for user `e0d5ff60d4b24fe0bbd771e8ed97db2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:54:20.389508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:20.450328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1217/2412] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:54:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:54:20.669249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf411c-1092-4688-aea4-358b76f87c76 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:20.670077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf411c-1092-4688-aea4-358b76f87c76 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:20.670077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf411c-1092-4688-aea4-358b76f87c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:20.689051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-25bf411c-1092-4688-aea4-358b76f87c76 None None] MFA Rules not processed for user `922f49fa7d324ebdade043e1be1a476f`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:54:20.690301 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-25bf411c-1092-4688-aea4-358b76f87c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:20.697849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1196/2413] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:54:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:54:25.225848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:54:25.227868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:25.317823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65b471-f7d7-41cd-aa2b-46c9ff7a7f00 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:54:25.321471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:54:25.322100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:25.322367 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:25.322804 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:25.326177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:25.408624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:54:25.410186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:54:25.420613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1218/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:54:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:54:28.278481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4baac-87fe-4cba-846a-dd06cab71b60 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:28.279013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4baac-87fe-4cba-846a-dd06cab71b60 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:28.279013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4baac-87fe-4cba-846a-dd06cab71b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:28.302289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-95f4baac-87fe-4cba-846a-dd06cab71b60 None None] MFA Rules not processed for user `58e209063f5c454d87d9f743bedfc734`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:54:28.303684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-95f4baac-87fe-4cba-846a-dd06cab71b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:54:28.365952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1197/2415] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:54:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:54:35.415983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=75 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:54:38.358176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=28 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:54:55.620683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:54:55.639969 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1481a9b-9ab5-4063-949d-859667f7fb18 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:54:55.644424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:54:55.645019 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:54:55.645290 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:54:55.645554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:54:55.656228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:54:55.657686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:54:55.668203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1219/2416] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:54:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:05.663956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=6 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:12.306490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4baac-87fe-4cba-846a-dd06cab71b60 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:12.321455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4baac-87fe-4cba-846a-dd06cab71b60 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:12.327759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52714d8a-2c58-4821-87e7-083e2635473d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:12.328692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52714d8a-2c58-4821-87e7-083e2635473d service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:12.329047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52714d8a-2c58-4821-87e7-083e2635473d service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:12.329359 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52714d8a-2c58-4821-87e7-083e2635473d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:12.331434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-52714d8a-2c58-4821-87e7-083e2635473d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:12.346903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52714d8a-2c58-4821-87e7-083e2635473d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:12.348456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52714d8a-2c58-4821-87e7-083e2635473d service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:12.360378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1198/2417] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:13.799006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:13.808297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dbe7d-3301-435d-8b0f-1147312c1b6b service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:13.812887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:13.813589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:13.813928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:13.814259 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:13.816779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:13.833161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:13.834611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:13.847061 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1220/2418] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:18.238567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457850-c65a-43d0-8164-0ca48ba6b9f1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:18.239101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457850-c65a-43d0-8164-0ca48ba6b9f1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:18.239101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457850-c65a-43d0-8164-0ca48ba6b9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:18.259353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-24457850-c65a-43d0-8164-0ca48ba6b9f1 None None] MFA Rules not processed for user `c0711a144b4c4432ab849367d3bd2b33`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:18.260547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-24457850-c65a-43d0-8164-0ca48ba6b9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:18.317900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1199/2419] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:20.838751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:20.841076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:20.923607 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70168-98ea-410d-ab4e-fce831f7a60c service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:20.935217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:20.935873 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:20.936144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:20.936443 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] PATH_INFO: `/v3/users/7bd68057dba244cabdb8efe86ee31b7e` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:20.937575 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd68057dba244cabdb8efe86ee31b7e)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:20.939544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:20.990932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd68057dba244cabdb8efe86ee31b7e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:20.991782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd68057dba244cabdb8efe86ee31b7e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:20.992461 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd68057dba244cabdb8efe86ee31b7e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.000936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd68057dba244cabdb8efe86ee31b7e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.011220 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9c936143-15ae-4936-9612-376f83c09dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd68057dba244cabdb8efe86ee31b7e'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.031243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1221/2420] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:55:20 2022] DELETE /identity/v3/users/7bd68057dba244cabdb8efe86ee31b7e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:55:21.225508 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457850-c65a-43d0-8164-0ca48ba6b9f1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:21.259164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457850-c65a-43d0-8164-0ca48ba6b9f1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:21.355178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:21.355900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.356253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.356709 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] PATH_INFO: `/v3/projects/ce501b5e17df4518a2ec76763bc3d81e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.357994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce501b5e17df4518a2ec76763bc3d81e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:21.360048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:21.416185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce501b5e17df4518a2ec76763bc3d81e', 'request_id': 'req-f4a44131-8eb1-4b57-b0a3-34e27bf91573'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.422868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce501b5e17df4518a2ec76763bc3d81e', 'request_id': 'req-f4a44131-8eb1-4b57-b0a3-34e27bf91573'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.437748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce501b5e17df4518a2ec76763bc3d81e', 'request_id': 'req-f4a44131-8eb1-4b57-b0a3-34e27bf91573'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.455543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] The token cache is being invalidate because project ce501b5e17df4518a2ec76763bc3d81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:55:21.455880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.457694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f4a44131-8eb1-4b57-b0a3-34e27bf91573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:21.462477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1200/2421] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:55:21 2022] DELETE /identity/v3/projects/ce501b5e17df4518a2ec76763bc3d81e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:55:21.464676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21560e7d-4372-4783-8c39-075b0ca5c003 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.465629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21560e7d-4372-4783-8c39-075b0ca5c003 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.465629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21560e7d-4372-4783-8c39-075b0ca5c003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.489392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-21560e7d-4372-4783-8c39-075b0ca5c003 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:21.490961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-21560e7d-4372-4783-8c39-075b0ca5c003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:21.555655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1222/2422] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:21.563857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0851d-5143-4b48-b154-115f963978d3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.564138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0851d-5143-4b48-b154-115f963978d3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.564444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0851d-5143-4b48-b154-115f963978d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.584303 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-dee0851d-5143-4b48-b154-115f963978d3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:21.585516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-dee0851d-5143-4b48-b154-115f963978d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:21.593565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1201/2423] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:21.597941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21560e7d-4372-4783-8c39-075b0ca5c003 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:21.607676 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21560e7d-4372-4783-8c39-075b0ca5c003 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:21.612195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:21.613018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.613319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.613674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.614320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:55:21.614620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:21.616229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:21.617820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:21.632434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1223/2424] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:55:21.635085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0851d-5143-4b48-b154-115f963978d3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:21.644534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0851d-5143-4b48-b154-115f963978d3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:21.650147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:21.650753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.651018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.651277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.651903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:21.654077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:21.672112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:21.687759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1202/2425] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:55:21 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:55:21.692045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:21.702604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d0345-264c-4328-b980-9b62619906d3 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:21.769515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:21.770218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.770508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.770759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.771376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:21.773687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:21.797475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1224/2426] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:55:21 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:55:21.801938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:21.813011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47a61-e880-4f07-b5c8-d5f4cf479abc admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:21.818235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:21.818847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.819102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.819419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.820028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:21.821688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:21.834745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1203/2427] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:55:21.839363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:21.850141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdc71e-6455-4e90-b064-0ee249fdb6a0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:21.857131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997ae86-83ff-4f08-8e8a-ad4b38aaf55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:21.857131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ae86-83ff-4f08-8e8a-ad4b38aaf55a admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.857131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ae86-83ff-4f08-8e8a-ad4b38aaf55a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.857131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ae86-83ff-4f08-8e8a-ad4b38aaf55a admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.857701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997ae86-83ff-4f08-8e8a-ad4b38aaf55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:21.859522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997ae86-83ff-4f08-8e8a-ad4b38aaf55a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:21.873193 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1225/2428] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:55:21.877620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:21.889360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a1356-af8d-4964-8880-25d594eb7664 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:21.897223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:21.898712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.899081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.899535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] PATH_INFO: `/v3/projects/9423dfed4b1b428d98d981b3d5ba73e5/users/8696a695f4b24375bb87f464cdf43787/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.916927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9423dfed4b1b428d98d981b3d5ba73e5, user_id=8696a695f4b24375bb87f464cdf43787, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:21.919748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:21.935965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1204/2429] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:55:21 2022] PUT /identity/v3/projects/9423dfed4b1b428d98d981b3d5ba73e5/users/8696a695f4b24375bb87f464cdf43787/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:55:21.950019 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429967b-a6ba-4528-b658-956518a410f6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:21.950301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429967b-a6ba-4528-b658-956518a410f6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:21.950590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429967b-a6ba-4528-b658-956518a410f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:21.984610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7429967b-a6ba-4528-b658-956518a410f6 None None] MFA Rules not processed for user `8696a695f4b24375bb87f464cdf43787`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:21.985953 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7429967b-a6ba-4528-b658-956518a410f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:22.076415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1226/2430] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:22.085874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:22.088607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:22.155938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfd4b5-4a63-4686-873e-7a653546cadc admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:22.165855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:22.166487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:22.166795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:22.167102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:22.179000 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:22.180518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:22.190999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1205/2431] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:25.537614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429967b-a6ba-4528-b658-956518a410f6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:25.548291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429967b-a6ba-4528-b658-956518a410f6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:25.553146 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:25.553770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:25.554046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:25.554309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:25.556253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:25.618595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:25.620018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:25.624598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:25.630497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1227/2432] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:32.185960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:33.376145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:33.378475 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:33.452860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c75a2-e6e4-4d4f-81b7-b882938a329c service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:33.457358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:33.457943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:33.458208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:33.458471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:33.461090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:33.519786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:33.521236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:33.530979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1206/2433] 10.222.0.23 () {60 vars in 1396 bytes} [Fri Aug 12 10:55:33 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:38.686921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:38.689542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:38.703985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ef5e1-f3bf-4d6c-9c1c-9e1723c91f73 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:38.708507 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f02110-fbde-4f2b-aecb-83225e753f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:38.709141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f02110-fbde-4f2b-aecb-83225e753f45 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:38.710022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f02110-fbde-4f2b-aecb-83225e753f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:38.710439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f02110-fbde-4f2b-aecb-83225e753f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:38.712727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-83f02110-fbde-4f2b-aecb-83225e753f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:38.727795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f02110-fbde-4f2b-aecb-83225e753f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:38.731831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f02110-fbde-4f2b-aecb-83225e753f45 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:38.747036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1228/2434] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:39.779934 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:39.847669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb7cfb-d7d8-4b71-95d3-ac7d048de803 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:39.854114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:39.854659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:39.854963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:39.855253 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] PATH_INFO: `/v3/users/8696a695f4b24375bb87f464cdf43787` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:39.856548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8696a695f4b24375bb87f464cdf43787)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:39.858534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:39.909441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8696a695f4b24375bb87f464cdf43787'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:39.916379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8696a695f4b24375bb87f464cdf43787'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:39.917058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8696a695f4b24375bb87f464cdf43787'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:39.918685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:39.926124 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8696a695f4b24375bb87f464cdf43787'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:39.939391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8696a695f4b24375bb87f464cdf43787'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:39.946147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1207/2435] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:55:39 2022] DELETE /identity/v3/users/8696a695f4b24375bb87f464cdf43787 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:55:39.953000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:39.953496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:39.953604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:39.975345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:39.976613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:40.038024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1229/2436] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:40.053919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:40.055556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:40.115917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b8b15-015a-4653-944c-4ab49a7b5f1f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:40.119863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:40.120493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:40.120762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:40.121024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:40.130729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:40.132577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:40.141679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1208/2437] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:40.425270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:40.426805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:40.439304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbfc9c-9bc4-4739-b173-7ac841a1e088 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:40.444042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:40.444981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:40.445257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:40.445669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] PATH_INFO: `/v3/projects/9423dfed4b1b428d98d981b3d5ba73e5` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:40.446801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9423dfed4b1b428d98d981b3d5ba73e5)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:40.448852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:40.501497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9423dfed4b1b428d98d981b3d5ba73e5', 'request_id': 'req-4f78defe-225c-42be-ba6c-c551273ad32d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:40.509396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9423dfed4b1b428d98d981b3d5ba73e5', 'request_id': 'req-4f78defe-225c-42be-ba6c-c551273ad32d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:40.515021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9423dfed4b1b428d98d981b3d5ba73e5', 'request_id': 'req-4f78defe-225c-42be-ba6c-c551273ad32d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:40.529200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] The token cache is being invalidate because project 9423dfed4b1b428d98d981b3d5ba73e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:55:40.529483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:40.530105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:40.531603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1230/2438] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:55:40 2022] DELETE /identity/v3/projects/9423dfed4b1b428d98d981b3d5ba73e5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:55:43.062930 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:43.065101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:43.150132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80a416-326b-45be-a7d7-dc939d164077 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:43.221078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:43.221688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:43.221970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:43.222241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:43.224104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:43.234767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:43.288827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:43.290480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:43.300297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1209/2439] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:50.526199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:51.261729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:51.266488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:51.287203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78defe-225c-42be-ba6c-c551273ad32d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:51.295392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:51.296007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:51.296229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:51.296551 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:51.298662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:51.316199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:51.317851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:51.328774 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1231/2440] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:52.961643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:52.963247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:53.024981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde6619-c092-4c99-a9cb-872e6b163137 service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:53.028871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:53.029419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:53.029681 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:53.029949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:53.031696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:53.096688 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:53.098167 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:53.108004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1210/2441] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:56.969174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:56.970582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:57.086168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:57.090744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:57.104304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a12d-4c9b-4659-8c81-1575c7a8bc8f service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:57.105831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580394c4-ae08-4a56-ac26-6a379485c0be service nova] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:57.109171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:57.109801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.110074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.110517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] PATH_INFO: `/v3/users/58e209063f5c454d87d9f743bedfc734` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.110966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:57.111592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.111796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e209063f5c454d87d9f743bedfc734)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:57.111915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.112113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.113813 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:57.114055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:57.128667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:57.130269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:57.142520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1232/2442] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:55:57.187529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e209063f5c454d87d9f743bedfc734'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.194427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e209063f5c454d87d9f743bedfc734'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.195079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e209063f5c454d87d9f743bedfc734'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.196947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:57.204460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e209063f5c454d87d9f743bedfc734'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.217553 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-372b218b-40a7-4000-aef6-dec0c0031257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e209063f5c454d87d9f743bedfc734'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.227039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1211/2443] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:55:57 2022] DELETE /identity/v3/users/58e209063f5c454d87d9f743bedfc734 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:55:57.442687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:57.463280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d80e0-6a02-44ea-8a53-c93ba27cd780 service placement] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:57.543766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:57.544034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.544364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.544614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] PATH_INFO: `/v3/projects/dc63f648a390465e94ab8170362a9b01` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.545806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc63f648a390465e94ab8170362a9b01)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:57.547896 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:57.604938 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc63f648a390465e94ab8170362a9b01', 'request_id': 'req-4abea570-1221-447d-a26b-b1792ff114fa'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.611355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc63f648a390465e94ab8170362a9b01', 'request_id': 'req-4abea570-1221-447d-a26b-b1792ff114fa'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.617674 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc63f648a390465e94ab8170362a9b01', 'request_id': 'req-4abea570-1221-447d-a26b-b1792ff114fa'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.630608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] The token cache is being invalidate because project dc63f648a390465e94ab8170362a9b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:55:57.630973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.631593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4abea570-1221-447d-a26b-b1792ff114fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:55:57.633309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1233/2444] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:55:57 2022] DELETE /identity/v3/projects/dc63f648a390465e94ab8170362a9b01 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:55:57.638472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509d1e-70ed-4734-a257-1f105ea52412 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.638816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509d1e-70ed-4734-a257-1f105ea52412 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.639151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce509d1e-70ed-4734-a257-1f105ea52412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.659304 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-ce509d1e-70ed-4734-a257-1f105ea52412 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:57.660544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ce509d1e-70ed-4734-a257-1f105ea52412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:57.722517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1212/2445] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:57.730622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae9934-1b05-4969-9fdb-9816e7b9f68f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.730906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae9934-1b05-4969-9fdb-9816e7b9f68f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.731272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae9934-1b05-4969-9fdb-9816e7b9f68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.746710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:57.752689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-10ae9934-1b05-4969-9fdb-9816e7b9f68f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:57.754561 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-10ae9934-1b05-4969-9fdb-9816e7b9f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:57.764909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1234/2446] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:57.769472 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509d1e-70ed-4734-a257-1f105ea52412 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:57.780024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce509d1e-70ed-4734-a257-1f105ea52412 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:57.784599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:57.785327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.785598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.785886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.786671 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:55:57.786994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:57.788690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:57.802345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1213/2447] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:55:57.806735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae9934-1b05-4969-9fdb-9816e7b9f68f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:57.816916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae9934-1b05-4969-9fdb-9816e7b9f68f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:57.822282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:57.822901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.823195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.823446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.824093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:57.826121 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:57.862320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1235/2448] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:55:57 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:55:57.866840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:57.878219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87ab88-e5b9-4bbc-8767-257cc78bc22e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:57.947523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:57.948119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:57.948430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:57.948698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:57.949335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:57.951510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:57.978234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1214/2449] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:55:57 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:55:57.982242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:57.994547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84664ed-b575-4b50-b78f-d785bc39f561 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:58.000492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:58.001106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:58.001376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:58.001776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:58.002416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:58.004507 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:58.017123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1236/2450] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:55:58.021235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:58.032919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3328a1b-475f-4713-9816-d8087008debc admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:58.037805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ecb2d-90fc-4062-bf06-361d6c240d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:58.038427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb2d-90fc-4062-bf06-361d6c240d7e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:58.038707 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb2d-90fc-4062-bf06-361d6c240d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:58.038977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb2d-90fc-4062-bf06-361d6c240d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:58.039620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecb2d-90fc-4062-bf06-361d6c240d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:58.041313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecb2d-90fc-4062-bf06-361d6c240d7e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:58.053198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1215/2451] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:55:58.057752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:58.068870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7621-b699-44fe-9d87-c3dd424b9df0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:58.073626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:58.074410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:58.074780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:58.075057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] PATH_INFO: `/v3/projects/b93ff276e6894c77a231ef075b02ca9a/users/5cde581444394ae0b342d0c1ec435c76/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:58.093977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93ff276e6894c77a231ef075b02ca9a, user_id=5cde581444394ae0b342d0c1ec435c76, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:58.097152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:58.117930 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1237/2452] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:55:58 2022] PUT /identity/v3/projects/b93ff276e6894c77a231ef075b02ca9a/users/5cde581444394ae0b342d0c1ec435c76/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:55:58.121264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58337048-f68e-4d2e-b3ac-bf2e4c3a1c92 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:58.121750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58337048-f68e-4d2e-b3ac-bf2e4c3a1c92 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:58.121989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58337048-f68e-4d2e-b3ac-bf2e4c3a1c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:58.142193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-58337048-f68e-4d2e-b3ac-bf2e4c3a1c92 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:55:58.143490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-58337048-f68e-4d2e-b3ac-bf2e4c3a1c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:58.185905 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:55:58.215952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1216/2453] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:55:58.226080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:55:58.228393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:55:58.299875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248fb44-fe87-4c20-8c40-e62b3a611b51 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:55:58.305373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:55:58.306314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:55:58.306684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:55:58.306994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:55:58.318355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:55:58.319825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:55:58.330695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1238/2454] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:56:00.228953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:00.229497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:00.229497 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:00.249159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] MFA Rules not processed for user `e0d5ff60d4b24fe0bbd771e8ed97db2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:56:00.250258 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:00.310267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1217/2455] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:56:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:56:01.083790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:01.099076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d77adf-cf85-4020-b130-45e0264723f6 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:01.104332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f507f50-1015-4fca-9344-7f14f213f7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:01.104959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f507f50-1015-4fca-9344-7f14f213f7da service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:01.105298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f507f50-1015-4fca-9344-7f14f213f7da service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:01.105511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f507f50-1015-4fca-9344-7f14f213f7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:01.107584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2f507f50-1015-4fca-9344-7f14f213f7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:01.128191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f507f50-1015-4fca-9344-7f14f213f7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:01.129844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f507f50-1015-4fca-9344-7f14f213f7da service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:01.138611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1239/2456] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:56:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:56:01.567900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:01.569920 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:01.671401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44822f-cf0e-42c8-ab17-8e0a9bb1bcca None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:01.676335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:01.677040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:01.677380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:01.677716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:01.679699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:01.692615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:01.694358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:01.705486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1218/2457] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:56:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:56:03.354154 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046be880-134a-4c48-b9c9-511e43206918 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:03.355672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046be880-134a-4c48-b9c9-511e43206918 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:03.356307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046be880-134a-4c48-b9c9-511e43206918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:03.386292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-046be880-134a-4c48-b9c9-511e43206918 None None] MFA Rules not processed for user `5cde581444394ae0b342d0c1ec435c76`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:56:03.387894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-046be880-134a-4c48-b9c9-511e43206918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:03.460724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1240/2458] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:56:03.476704 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:03.478762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:03.605565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260462a5-b1fb-423e-8bc3-925cbd65833c service placement] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:03.612650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:03.613457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:03.613789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:03.614120 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:03.626106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:03.627886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:03.634320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:56:03.641307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1219/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:56:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:56:05.299235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da83fc-2c06-4f16-855c-f2d11af3e154 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:05.299759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da83fc-2c06-4f16-855c-f2d11af3e154 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:05.299759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da83fc-2c06-4f16-855c-f2d11af3e154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:05.321180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-b2da83fc-2c06-4f16-855c-f2d11af3e154 None None] MFA Rules not processed for user `5cde581444394ae0b342d0c1ec435c76`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:56:05.322671 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b2da83fc-2c06-4f16-855c-f2d11af3e154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:05.330390 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1241/2460] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:56:05.647233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:05.657856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca776622-30e5-422f-a1a1-66f77e8a1967 service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:05.661735 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:05.662982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:05.663260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:05.663532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:05.665370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:05.675477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:05.676959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:05.685328 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1220/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:56:09.230736 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da83fc-2c06-4f16-855c-f2d11af3e154 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:09.246904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da83fc-2c06-4f16-855c-f2d11af3e154 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:09.252719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f8bf6-dffe-4c50-9dd0-cb4fa6c9cdef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:09.253291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8bf6-dffe-4c50-9dd0-cb4fa6c9cdef service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:09.253814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8bf6-dffe-4c50-9dd0-cb4fa6c9cdef service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:09.254060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8bf6-dffe-4c50-9dd0-cb4fa6c9cdef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:09.255882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-437f8bf6-dffe-4c50-9dd0-cb4fa6c9cdef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:09.269349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f8bf6-dffe-4c50-9dd0-cb4fa6c9cdef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:09.271116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f8bf6-dffe-4c50-9dd0-cb4fa6c9cdef service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:09.283486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1242/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:56:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:56:13.411540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:13.425915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf3e3c-1eba-43f1-a0b5-ab8026b9bbdf service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:13.431609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:13.432635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:13.433100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:13.433427 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:13.435831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:13.450002 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:13.452543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:13.469822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1221/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:56:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:56:19.279348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:56:23.459761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=12 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:56:23.712158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:23.712672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:23.713158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:23.733400 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:56:23.734701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:23.802210 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1243/2464] 10.222.0.23 () {58 vars in 967 bytes} [Fri Aug 12 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:56:23.857453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:23.859703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:23.937431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343614b6-def9-4e93-aece-c0fdecfbe2f6 service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:23.943638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:23.944246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:23.944549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:23.944816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:23.954390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:23.957014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:23.969752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1222/2465] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:56:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:56:31.157340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:31.158851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:31.174699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863f125-0664-44c5-a5c1-23fe892340bc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:31.182516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:31.183202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:31.183474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:31.183733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] PATH_INFO: `/v3/users/3f40eebcea60459ab0cf9233aea9b743` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:31.184992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f40eebcea60459ab0cf9233aea9b743)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:31.187206 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:31.235990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f40eebcea60459ab0cf9233aea9b743'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.236643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f40eebcea60459ab0cf9233aea9b743'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.237247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f40eebcea60459ab0cf9233aea9b743'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.243837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f40eebcea60459ab0cf9233aea9b743'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.252928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f40eebcea60459ab0cf9233aea9b743'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.274872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1244/2466] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:56:31 2022] DELETE /identity/v3/users/3f40eebcea60459ab0cf9233aea9b743 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:56:31.465875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:31.480717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f8f44-5408-4114-b662-bde927e3a2a9 service swift] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:31.546199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:31.546870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:31.547144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:31.547406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] PATH_INFO: `/v3/projects/807c809e71de43ebb4edecbab4bc4292` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:31.548585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=807c809e71de43ebb4edecbab4bc4292)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:31.550494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:31.598240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807c809e71de43ebb4edecbab4bc4292', 'request_id': 'req-a5091c0c-554d-4cf8-a222-24cc8bbb7508'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.603666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807c809e71de43ebb4edecbab4bc4292', 'request_id': 'req-a5091c0c-554d-4cf8-a222-24cc8bbb7508'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.610481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807c809e71de43ebb4edecbab4bc4292', 'request_id': 'req-a5091c0c-554d-4cf8-a222-24cc8bbb7508'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.624801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] The token cache is being invalidate because project 807c809e71de43ebb4edecbab4bc4292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:56:31.625090 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.625945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.628185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1223/2467] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:56:31 2022] DELETE /identity/v3/projects/807c809e71de43ebb4edecbab4bc4292 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:56:31.632317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:31.633854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:31.646494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec92c96-4133-4d81-895a-7bf249ceba3a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:31.703403 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:31.704034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:31.704328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:31.704595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] PATH_INFO: `/v3/users/51b6307613b7496eabcf12273a4db967` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:31.705728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b6307613b7496eabcf12273a4db967)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:31.707634 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:31.728450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:56:31.759669 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b6307613b7496eabcf12273a4db967'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.760344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b6307613b7496eabcf12273a4db967'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.760948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b6307613b7496eabcf12273a4db967'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.768559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b6307613b7496eabcf12273a4db967'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.778130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d3a3e11c-eb51-478c-84a4-3f782c912222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b6307613b7496eabcf12273a4db967'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:31.798794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1245/2468] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:56:31 2022] DELETE /identity/v3/users/51b6307613b7496eabcf12273a4db967 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:56:32.014519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:32.016367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:32.030176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5091c0c-554d-4cf8-a222-24cc8bbb7508 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:32.059286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:56:32.121960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:32.122605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:32.122878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:32.123139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] PATH_INFO: `/v3/projects/9d829e430cf44f5882e1ccc2f7cb5c06` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:32.124285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d829e430cf44f5882e1ccc2f7cb5c06)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:32.126436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:32.173756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d829e430cf44f5882e1ccc2f7cb5c06', 'request_id': 'req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:32.179315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d829e430cf44f5882e1ccc2f7cb5c06', 'request_id': 'req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:32.188450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d829e430cf44f5882e1ccc2f7cb5c06', 'request_id': 'req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:32.201984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] The token cache is being invalidate because project 9d829e430cf44f5882e1ccc2f7cb5c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:56:32.202437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:32.203614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:56:32.205407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1224/2469] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:56:32 2022] DELETE /identity/v3/projects/9d829e430cf44f5882e1ccc2f7cb5c06 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:56:37.862686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa00d9-6104-491e-95f6-40c8d616b1c9 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:37.863260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa00d9-6104-491e-95f6-40c8d616b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:37.863260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa00d9-6104-491e-95f6-40c8d616b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:37.885506 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-cfaa00d9-6104-491e-95f6-40c8d616b1c9 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:56:37.886725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-cfaa00d9-6104-491e-95f6-40c8d616b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:37.946683 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1246/2470] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:56:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:56:37.959793 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:37.961462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:38.032007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b491-5f86-4895-b5a2-cf0a3708b3d2 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:38.037154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:38.037792 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:38.038066 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:38.038372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:38.048130 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:38.049824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:38.058261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1225/2471] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:56:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:56:47.939584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=8,keystone:SELECT=45,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:56:48.055192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:56:53.224280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4e09-ed3e-4f30-9748-cec9c73e6941 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:53.224740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4e09-ed3e-4f30-9748-cec9c73e6941 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:53.224846 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4e09-ed3e-4f30-9748-cec9c73e6941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:53.244768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ddbc4e09-ed3e-4f30-9748-cec9c73e6941 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:56:53.245988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ddbc4e09-ed3e-4f30-9748-cec9c73e6941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:56:53.253972 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1247/2472] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Aug 12 10:56:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:56:53.266348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:56:53.277573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53510b1-f061-4bf6-a1a3-1087524aaef8 service swift] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:56:53.282958 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:56:53.283700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:56:53.283969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:56:53.284381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:56:53.293078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:56:53.294705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:56:53.303255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1226/2473] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:56:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:57:03.239789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:03.300018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=6 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:06.994264 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:06.994779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:06.994929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:07.014753 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] MFA Rules not processed for user `e8008fa945f540a18aac7b0f81edf0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:07.016030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:07.024647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1248/2474] 10.222.0.23 () {58 vars in 967 bytes} [Fri Aug 12 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:07.037686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:07.050041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39860d0-476e-45cc-a1ab-170151dcdf90 service swift] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:07.056033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f145d-8375-43ea-97f9-32373869e8ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01afe498af8c42e8bca4e993897ad1d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:07.056668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f145d-8375-43ea-97f9-32373869e8ce service swift] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:07.056951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f145d-8375-43ea-97f9-32373869e8ce service swift] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:07.057221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f145d-8375-43ea-97f9-32373869e8ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:07.066032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f145d-8375-43ea-97f9-32373869e8ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:07.067692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f145d-8375-43ea-97f9-32373869e8ce service swift] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:07.076084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1227/2475] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Aug 12 10:57:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 10:57:17.009954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:17.072952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=6 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:19.827992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:19.830111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:19.891370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e4e3c-694c-419f-ae41-34a9006cbe04 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:19.897718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:19.898863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:19.899135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:19.899396 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:19.901600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:19.941923 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fa65-75fb-4c7e-b881-8639b8bde3b1 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:19.942397 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fa65-75fb-4c7e-b881-8639b8bde3b1 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:19.942559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fa65-75fb-4c7e-b881-8639b8bde3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:19.968655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:19.969311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-abb3fa65-75fb-4c7e-b881-8639b8bde3b1 None None] MFA Rules not processed for user `922f49fa7d324ebdade043e1be1a476f`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:19.970110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:19.970520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-abb3fa65-75fb-4c7e-b881-8639b8bde3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:19.980533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1249/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:20.031042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1228/2477] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:24.594176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:24.595830 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:24.609946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166a2b6-ef39-4b2c-93aa-3b6fd85f16e8 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:24.615349 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:24.617046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:24.617783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:24.618225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] PATH_INFO: `/v3/users/e0d5ff60d4b24fe0bbd771e8ed97db2f` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:24.620226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d5ff60d4b24fe0bbd771e8ed97db2f)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:24.625751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:24.719459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d5ff60d4b24fe0bbd771e8ed97db2f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:24.720135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d5ff60d4b24fe0bbd771e8ed97db2f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:24.720780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d5ff60d4b24fe0bbd771e8ed97db2f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:24.729539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d5ff60d4b24fe0bbd771e8ed97db2f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:24.742552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0ea9565e-1606-428c-a10f-0286e92dadc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d5ff60d4b24fe0bbd771e8ed97db2f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:24.770520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1250/2478] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:24 2022] DELETE /identity/v3/users/e0d5ff60d4b24fe0bbd771e8ed97db2f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:24.999940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3fa65-75fb-4c7e-b881-8639b8bde3b1 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.011441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3fa65-75fb-4c7e-b881-8639b8bde3b1 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.070267 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.070907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.071182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.071469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] PATH_INFO: `/v3/projects/362a04e0181b4fa49e27fb38c04ac677` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.072708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=362a04e0181b4fa49e27fb38c04ac677)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.075373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.124821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362a04e0181b4fa49e27fb38c04ac677', 'request_id': 'req-87ed169c-4770-43d9-b75c-017ac01825d5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:25.130333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362a04e0181b4fa49e27fb38c04ac677', 'request_id': 'req-87ed169c-4770-43d9-b75c-017ac01825d5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:25.136826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362a04e0181b4fa49e27fb38c04ac677', 'request_id': 'req-87ed169c-4770-43d9-b75c-017ac01825d5'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:25.156131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] The token cache is being invalidate because project 362a04e0181b4fa49e27fb38c04ac677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:25.156467 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:25.157142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-87ed169c-4770-43d9-b75c-017ac01825d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:25.158836 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1229/2479] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:24 2022] DELETE /identity/v3/projects/362a04e0181b4fa49e27fb38c04ac677 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:25.163672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d372c-f4b0-49ff-bda4-4d273758fd67 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.163980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d372c-f4b0-49ff-bda4-4d273758fd67 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.164279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d372c-f4b0-49ff-bda4-4d273758fd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.189579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-698d372c-f4b0-49ff-bda4-4d273758fd67 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:25.190916 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-698d372c-f4b0-49ff-bda4-4d273758fd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:25.218521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:25.276468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1251/2480] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:25.284629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef0c9d-eed3-4b5d-83f6-650e67d79ec8 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.284909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef0c9d-eed3-4b5d-83f6-650e67d79ec8 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.285350 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef0c9d-eed3-4b5d-83f6-650e67d79ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.307250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-62ef0c9d-eed3-4b5d-83f6-650e67d79ec8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:25.308794 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-62ef0c9d-eed3-4b5d-83f6-650e67d79ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:25.317333 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1230/2481] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:25.321528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d372c-f4b0-49ff-bda4-4d273758fd67 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.344320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d372c-f4b0-49ff-bda4-4d273758fd67 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.348812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.349426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.349701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.349971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.350699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:25.351000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.352656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.368081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1252/2482] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:57:25.373482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef0c9d-eed3-4b5d-83f6-650e67d79ec8 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.383378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef0c9d-eed3-4b5d-83f6-650e67d79ec8 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.388807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.389412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.389710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.389975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.390597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.392549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.423751 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1231/2483] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:25.428014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.438631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca6dea-4b58-4518-96de-404046ba8513 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.497679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.498317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.498572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.498835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.499525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.501565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.525457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1253/2484] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:25 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:25.529917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.540765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30781ed-8a9c-41b0-aa61-5b8e29b9fafa admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.546260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.546867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.547344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.547612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.548293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.549914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.551314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:25.563942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1232/2485] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:25.567995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.578114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8e36-57fb-4a83-ae4d-28a4f65b1794 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.582936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5d64d-c539-4514-a930-63571b3d30d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.583538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5d64d-c539-4514-a930-63571b3d30d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.583813 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5d64d-c539-4514-a930-63571b3d30d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.584080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5d64d-c539-4514-a930-63571b3d30d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.584766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5d64d-c539-4514-a930-63571b3d30d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.586370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5d64d-c539-4514-a930-63571b3d30d3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.597245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1254/2486] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:25.601498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.611745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e528-ba4b-47c0-bbc4-acb9ade41ce3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.616991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.617811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.618165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.618503 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] PATH_INFO: `/v3/projects/9cb79dff5be04fa19da68eadb8670634/users/9adfa9cd417445758574cfc1cda5f36c/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.633632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb79dff5be04fa19da68eadb8670634, user_id=9adfa9cd417445758574cfc1cda5f36c, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.636380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.652917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1233/2487] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:25 2022] PUT /identity/v3/projects/9cb79dff5be04fa19da68eadb8670634/users/9adfa9cd417445758574cfc1cda5f36c/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:25.663385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8fbb-af66-47fb-bbf7-09fd817c422e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.663668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8fbb-af66-47fb-bbf7-09fd817c422e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.663956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8fbb-af66-47fb-bbf7-09fd817c422e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.693684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-6e7d8fbb-af66-47fb-bbf7-09fd817c422e None None] MFA Rules not processed for user `9adfa9cd417445758574cfc1cda5f36c`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:25.694912 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6e7d8fbb-af66-47fb-bbf7-09fd817c422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:25.765440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1255/2488] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:25.770015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.843048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca4d72-4ab5-455c-964d-df5902b718f8 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.848134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.848825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.849094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.849379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.850006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.851931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.880384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1234/2489] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:25.884488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8fbb-af66-47fb-bbf7-09fd817c422e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.896244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8fbb-af66-47fb-bbf7-09fd817c422e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:25.910269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:25.958595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:25.959337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:25.959613 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:25.959880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:25.960546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:25.962496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:25.983546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1256/2490] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:25 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:25.987696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:25.998291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac410-21a5-419e-867c-eff7e47634e6 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:26.003060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:26.003674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.003916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.004146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:26.004807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:26.006406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:26.016983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1235/2491] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:26.021151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:26.032452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bbdb8-b8f2-495b-be7b-5cf9ebf193e6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:26.038366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:26.039050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.039296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.039579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] PATH_INFO: `/v3/projects/bae357255c8e45749c56e4e4112bb841/users/92595e53c2e346b98161b0fbab308828/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:26.055439 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae357255c8e45749c56e4e4112bb841, user_id=92595e53c2e346b98161b0fbab308828, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:26.058151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:26.073106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1257/2492] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:26 2022] PUT /identity/v3/projects/bae357255c8e45749c56e4e4112bb841/users/92595e53c2e346b98161b0fbab308828/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:57:26.077679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:26.087916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e072f9-fb93-4978-9724-45cf9c6af10b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:26.148223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175b3f7-4040-4c66-9737-ec815ca5bdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:26.149191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175b3f7-4040-4c66-9737-ec815ca5bdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.149459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175b3f7-4040-4c66-9737-ec815ca5bdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.149725 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175b3f7-4040-4c66-9737-ec815ca5bdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:26.150357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175b3f7-4040-4c66-9737-ec815ca5bdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:26.151977 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175b3f7-4040-4c66-9737-ec815ca5bdc9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:26.162631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1236/2493] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:26.168366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:26.178887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48015a9-c5af-48d1-a909-d6bc1ab79d5a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:26.183748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:26.184426 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.184696 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.184962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] PATH_INFO: `/v3/projects/bae357255c8e45749c56e4e4112bb841/users/92595e53c2e346b98161b0fbab308828/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:26.186904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae357255c8e45749c56e4e4112bb841, user_id=92595e53c2e346b98161b0fbab308828, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:26.189585 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:26.199942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1258/2494] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:26 2022] PUT /identity/v3/projects/bae357255c8e45749c56e4e4112bb841/users/92595e53c2e346b98161b0fbab308828/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:26.212188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da30f8-66cd-4ac0-817e-40c42ccd1eb2 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.212509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da30f8-66cd-4ac0-817e-40c42ccd1eb2 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.212783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da30f8-66cd-4ac0-817e-40c42ccd1eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:26.242701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-73da30f8-66cd-4ac0-817e-40c42ccd1eb2 None None] MFA Rules not processed for user `92595e53c2e346b98161b0fbab308828`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:26.243918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-73da30f8-66cd-4ac0-817e-40c42ccd1eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:26.274559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:26.313404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1237/2495] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:26.325200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:26.327246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:26.388360 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cf32e-5b55-4fe7-960f-296baee45421 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:26.392067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b924a11-8af2-4f79-a889-ecc3f877bd64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:26.392701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b924a11-8af2-4f79-a889-ecc3f877bd64 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.392975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b924a11-8af2-4f79-a889-ecc3f877bd64 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.393240 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b924a11-8af2-4f79-a889-ecc3f877bd64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:26.403457 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b924a11-8af2-4f79-a889-ecc3f877bd64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:26.405052 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b924a11-8af2-4f79-a889-ecc3f877bd64 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:26.415568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1259/2496] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:26.820809 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da30f8-66cd-4ac0-817e-40c42ccd1eb2 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:26.882491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da30f8-66cd-4ac0-817e-40c42ccd1eb2 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:26.887119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:26.887750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.888054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.888313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] PATH_INFO: `/v3/users/9adfa9cd417445758574cfc1cda5f36c` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:26.889509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9adfa9cd417445758574cfc1cda5f36c)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:26.891444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:26.954163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9adfa9cd417445758574cfc1cda5f36c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:26.961690 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9adfa9cd417445758574cfc1cda5f36c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:26.962386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9adfa9cd417445758574cfc1cda5f36c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:26.971176 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9adfa9cd417445758574cfc1cda5f36c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:26.983914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9adfa9cd417445758574cfc1cda5f36c'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:26.990161 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1238/2497] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:26 2022] DELETE /identity/v3/users/9adfa9cd417445758574cfc1cda5f36c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:26.994973 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:26.995383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:26.995632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:27.015128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:27.016369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:27.076663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1260/2498] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:27.086465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:27.087890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:27.156214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:27.163034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7569ab-bfd0-454f-abc5-2e599c0d7395 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:27.168913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:27.169656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:27.170001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:27.170351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:27.183607 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:27.185530 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:27.197368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1239/2499] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:27.685785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:27.687479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:27.702432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07888576-2862-4d5b-aafa-d1ac3e32eae8 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:27.707941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:27.708900 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:27.709190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:27.709524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] PATH_INFO: `/v3/projects/9cb79dff5be04fa19da68eadb8670634` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:27.711368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb79dff5be04fa19da68eadb8670634)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:27.713843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:27.723120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:27.766927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb79dff5be04fa19da68eadb8670634', 'request_id': 'req-8197e4ad-b920-46cb-bf9c-c924f0f931c0'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.773237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb79dff5be04fa19da68eadb8670634', 'request_id': 'req-8197e4ad-b920-46cb-bf9c-c924f0f931c0'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.779508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb79dff5be04fa19da68eadb8670634', 'request_id': 'req-8197e4ad-b920-46cb-bf9c-c924f0f931c0'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.792685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] The token cache is being invalidate because project 9cb79dff5be04fa19da68eadb8670634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:27.792971 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.793577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.795079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1261/2500] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:27 2022] DELETE /identity/v3/projects/9cb79dff5be04fa19da68eadb8670634 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:27.799148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:27.808883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3dcfdf-b80c-42ea-bbe9-c7fa227a3de4 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:27.869806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:27.870391 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:27.870867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:27.871144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] PATH_INFO: `/v3/users/92595e53c2e346b98161b0fbab308828` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:27.872279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92595e53c2e346b98161b0fbab308828)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:27.874254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:27.926728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92595e53c2e346b98161b0fbab308828'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.933702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92595e53c2e346b98161b0fbab308828'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.934380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92595e53c2e346b98161b0fbab308828'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.942916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92595e53c2e346b98161b0fbab308828'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.955454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-598d4d15-7af6-4090-8bac-a57534fa7ae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92595e53c2e346b98161b0fbab308828'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:27.961430 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1240/2501] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:27 2022] DELETE /identity/v3/users/92595e53c2e346b98161b0fbab308828 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:28.256794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:28.258300 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:28.271347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e4ad-b920-46cb-bf9c-c924f0f931c0 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:28.338486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:28.339581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.339581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.339771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] PATH_INFO: `/v3/projects/bae357255c8e45749c56e4e4112bb841` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.341055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bae357255c8e45749c56e4e4112bb841)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:28.342962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:28.393874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bae357255c8e45749c56e4e4112bb841', 'request_id': 'req-af49a383-edf9-482e-8f0f-8c1740232a7f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:28.400414 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bae357255c8e45749c56e4e4112bb841', 'request_id': 'req-af49a383-edf9-482e-8f0f-8c1740232a7f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:28.406366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bae357255c8e45749c56e4e4112bb841', 'request_id': 'req-af49a383-edf9-482e-8f0f-8c1740232a7f'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:28.421657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] The token cache is being invalidate because project bae357255c8e45749c56e4e4112bb841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:28.421956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:28.422601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-af49a383-edf9-482e-8f0f-8c1740232a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:28.424482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1262/2502] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:28 2022] DELETE /identity/v3/projects/bae357255c8e45749c56e4e4112bb841 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:28.429183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0b0e-508b-4fa8-ab63-f6e0fd4a1230 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.429691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0b0e-508b-4fa8-ab63-f6e0fd4a1230 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.429691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0b0e-508b-4fa8-ab63-f6e0fd4a1230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.450345 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b07b0b0e-508b-4fa8-ab63-f6e0fd4a1230 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:28.451696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b07b0b0e-508b-4fa8-ab63-f6e0fd4a1230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:28.516850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1241/2503] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:28.524975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ce179-d847-4aa6-adee-9a81dadecc98 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.525257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ce179-d847-4aa6-adee-9a81dadecc98 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.525528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ce179-d847-4aa6-adee-9a81dadecc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.546868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-215ce179-d847-4aa6-adee-9a81dadecc98 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:28.548091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-215ce179-d847-4aa6-adee-9a81dadecc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:28.556337 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1263/2504] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:28.560889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b0b0e-508b-4fa8-ab63-f6e0fd4a1230 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:28.570626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b0b0e-508b-4fa8-ab63-f6e0fd4a1230 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:28.575490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:28.576077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.576383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.576646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.577286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:28.577577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:28.579164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:28.581438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:28.593098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1242/2505] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:57:28.597526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ce179-d847-4aa6-adee-9a81dadecc98 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:28.608047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ce179-d847-4aa6-adee-9a81dadecc98 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:28.613569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:28.614159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.614433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.614699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.615332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:28.617383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:28.642152 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:28.651182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1264/2506] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:28 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:28.655411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:28.665864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777ef63-1291-45a5-8240-981f82e75f0b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:28.729355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:28.729983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.730243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.730513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.731155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:28.733226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:28.758816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1243/2507] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:28 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:28.763091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:28.774189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8ee0-2c1d-43ee-9699-6886d2b7fe04 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:28.779342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:28.780042 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.780422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.780749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.781451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:28.783115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:28.794796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1265/2508] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:28.799092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:28.809692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af940-c312-4e7c-995f-a166afc6158c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:28.814602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef48135-6298-4ce8-bba9-703b7c02e030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:28.815208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef48135-6298-4ce8-bba9-703b7c02e030 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.815479 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef48135-6298-4ce8-bba9-703b7c02e030 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.815741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef48135-6298-4ce8-bba9-703b7c02e030 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.816406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef48135-6298-4ce8-bba9-703b7c02e030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:28.818043 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef48135-6298-4ce8-bba9-703b7c02e030 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:28.829412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1244/2509] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:57:28.833820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:28.844044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681204f0-fff1-4285-ae3c-06aad0ac0255 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:28.849627 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:28.850283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.850558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.850828 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] PATH_INFO: `/v3/projects/2d6db6b995f74e8f9ddd832bae1648f0/users/6d369d8409034d94ad534e6ab8f86eea/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.868117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6db6b995f74e8f9ddd832bae1648f0, user_id=6d369d8409034d94ad534e6ab8f86eea, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:28.871158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:28.889398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1266/2510] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:28 2022] PUT /identity/v3/projects/2d6db6b995f74e8f9ddd832bae1648f0/users/6d369d8409034d94ad534e6ab8f86eea/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:28.900467 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec328a-33fb-444b-be8e-68d31bac4864 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:28.900686 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec328a-33fb-444b-be8e-68d31bac4864 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:28.900955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec328a-33fb-444b-be8e-68d31bac4864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:28.930270 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-44ec328a-33fb-444b-be8e-68d31bac4864 None None] MFA Rules not processed for user `6d369d8409034d94ad534e6ab8f86eea`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:28.931487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-44ec328a-33fb-444b-be8e-68d31bac4864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:28.999060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1245/2511] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:29.003761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.067772 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a4c0d-e8fd-4de7-8045-0bcb3286a35b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.072821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.073448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.074088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.074363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.075014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.076949 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.108130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1267/2512] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:29 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:29.112636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec328a-33fb-444b-be8e-68d31bac4864 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.133343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec328a-33fb-444b-be8e-68d31bac4864 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.196726 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.197342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.197614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.197873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.198478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.200753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.210669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:29.225087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1246/2513] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:29 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:29.229246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.239802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9219e-249f-44c5-a59d-049502c5a8ec admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.244638 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.245221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.245499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.245991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.246810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.248430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.259221 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1268/2514] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:29.264687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.275213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393924c-3119-4c48-8e44-85f03dbbd79f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.280362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.281003 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.281274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.281545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] PATH_INFO: `/v3/projects/b4a91150958f4559a8eeed1e5a446cd0/users/2396e6c55b734a9ca9df20b7fdd908d4/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.296625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a91150958f4559a8eeed1e5a446cd0, user_id=2396e6c55b734a9ca9df20b7fdd908d4, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.299457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.314915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1247/2515] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:29 2022] PUT /identity/v3/projects/b4a91150958f4559a8eeed1e5a446cd0/users/2396e6c55b734a9ca9df20b7fdd908d4/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:29.319063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.329418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56da00a-2b61-4fe7-9ee9-59e18c66ef9a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.388515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d27913-717c-492c-a08f-0bf36efe3fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.389143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27913-717c-492c-a08f-0bf36efe3fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.389418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27913-717c-492c-a08f-0bf36efe3fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.389684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27913-717c-492c-a08f-0bf36efe3fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.390324 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d27913-717c-492c-a08f-0bf36efe3fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.391920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d27913-717c-492c-a08f-0bf36efe3fb4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.401944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1269/2516] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:29.408800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.420678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583f5c6-d809-408e-a291-4a3719978547 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.428986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.429964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.430539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.430539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] PATH_INFO: `/v3/projects/b4a91150958f4559a8eeed1e5a446cd0/users/2396e6c55b734a9ca9df20b7fdd908d4/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.434224 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a91150958f4559a8eeed1e5a446cd0, user_id=2396e6c55b734a9ca9df20b7fdd908d4, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.436972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.448672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1248/2517] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:29 2022] PUT /identity/v3/projects/b4a91150958f4559a8eeed1e5a446cd0/users/2396e6c55b734a9ca9df20b7fdd908d4/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:29.461389 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de6eb-d041-499e-8e3f-ef9aa618f42a None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.461688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de6eb-d041-499e-8e3f-ef9aa618f42a None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.461960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de6eb-d041-499e-8e3f-ef9aa618f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.491576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d51de6eb-d041-499e-8e3f-ef9aa618f42a None None] MFA Rules not processed for user `2396e6c55b734a9ca9df20b7fdd908d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:29.492810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d51de6eb-d041-499e-8e3f-ef9aa618f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:29.514711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:29.577054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1270/2518] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:29.588826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.591277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:29.652027 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e9f39-7390-4c46-996a-a0fb44130b2c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.655749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.656376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.656658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.656896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.666106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.667654 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.677344 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1249/2519] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:29.755135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de6eb-d041-499e-8e3f-ef9aa618f42a None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:29.765920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de6eb-d041-499e-8e3f-ef9aa618f42a None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:29.769977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4d909-7031-4fc9-9f2b-0d543f4a9f15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:29.770577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4d909-7031-4fc9-9f2b-0d543f4a9f15 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:29.770851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4d909-7031-4fc9-9f2b-0d543f4a9f15 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:29.771116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4d909-7031-4fc9-9f2b-0d543f4a9f15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:29.830983 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4d909-7031-4fc9-9f2b-0d543f4a9f15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:29.832459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4d909-7031-4fc9-9f2b-0d543f4a9f15 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:29.841237 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1271/2520] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:32.635386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:32.700949 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd307292-f9d9-4dbf-9fb8-29c9dc84ef01 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:32.706373 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:32.707009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:32.707285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:32.707554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] PATH_INFO: `/v3/users/6d369d8409034d94ad534e6ab8f86eea` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:32.708724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d369d8409034d94ad534e6ab8f86eea)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:32.710678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:32.762928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d369d8409034d94ad534e6ab8f86eea'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:32.769859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d369d8409034d94ad534e6ab8f86eea'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:32.770542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d369d8409034d94ad534e6ab8f86eea'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:32.774606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:32.779944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d369d8409034d94ad534e6ab8f86eea'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:32.793932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d369d8409034d94ad534e6ab8f86eea'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:32.800205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1250/2521] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:32 2022] DELETE /identity/v3/users/6d369d8409034d94ad534e6ab8f86eea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:32.804885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:32.805410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:32.805410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:32.825082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:32.826370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:32.887566 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1272/2522] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:32.897638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:32.899532 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:32.957611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0591e5a-f87f-4af9-a544-3c712715a7c7 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:32.961297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:32.962069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:32.962338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:32.962621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:32.972815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:32.974264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:32.985027 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1251/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:33.274320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:33.275759 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:33.288036 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08673f-9f44-4e45-a00b-87059533353b None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:33.293268 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:33.293895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:33.294169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:33.294449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] PATH_INFO: `/v3/projects/2d6db6b995f74e8f9ddd832bae1648f0` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:33.295610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6db6b995f74e8f9ddd832bae1648f0)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:33.297565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:33.353659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6db6b995f74e8f9ddd832bae1648f0', 'request_id': 'req-06e24053-1ed6-4f91-94a4-d883f0894597'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.357098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:33.363244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6db6b995f74e8f9ddd832bae1648f0', 'request_id': 'req-06e24053-1ed6-4f91-94a4-d883f0894597'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.369763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6db6b995f74e8f9ddd832bae1648f0', 'request_id': 'req-06e24053-1ed6-4f91-94a4-d883f0894597'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.384611 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] The token cache is being invalidate because project 2d6db6b995f74e8f9ddd832bae1648f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:33.384894 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.385496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.387074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1273/2524] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:33 2022] DELETE /identity/v3/projects/2d6db6b995f74e8f9ddd832bae1648f0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:33.391320 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:33.401347 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5373ead-6a95-43ce-85b1-5856a22cff1e service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:33.477799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:33.478486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:33.478762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:33.479033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] PATH_INFO: `/v3/users/2396e6c55b734a9ca9df20b7fdd908d4` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:33.480756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2396e6c55b734a9ca9df20b7fdd908d4)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:33.482892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:33.537207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2396e6c55b734a9ca9df20b7fdd908d4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.545074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2396e6c55b734a9ca9df20b7fdd908d4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.545783 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2396e6c55b734a9ca9df20b7fdd908d4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.555870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2396e6c55b734a9ca9df20b7fdd908d4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.573505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f56a289e-6526-4e9d-b28c-d18d99346e86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2396e6c55b734a9ca9df20b7fdd908d4'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:33.580822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:33.580822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1252/2525] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:33 2022] DELETE /identity/v3/users/2396e6c55b734a9ca9df20b7fdd908d4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:33.869705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:33.871663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:33.887810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e24053-1ed6-4f91-94a4-d883f0894597 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:33.951201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:33.951930 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:33.952224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:33.952539 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] PATH_INFO: `/v3/projects/b4a91150958f4559a8eeed1e5a446cd0` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:33.953673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a91150958f4559a8eeed1e5a446cd0)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:33.955579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.001361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a91150958f4559a8eeed1e5a446cd0', 'request_id': 'req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:34.007526 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a91150958f4559a8eeed1e5a446cd0', 'request_id': 'req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:34.013768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a91150958f4559a8eeed1e5a446cd0', 'request_id': 'req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:34.033080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] The token cache is being invalidate because project b4a91150958f4559a8eeed1e5a446cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:34.033435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:34.034563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-8ad4fc0c-dd83-4ed5-ae46-62d7919ee098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:34.036790 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1274/2526] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:33 2022] DELETE /identity/v3/projects/b4a91150958f4559a8eeed1e5a446cd0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:34.042676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420df45-7697-4078-82ea-1db406ce9369 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.043198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420df45-7697-4078-82ea-1db406ce9369 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.043198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420df45-7697-4078-82ea-1db406ce9369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.063324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a420df45-7697-4078-82ea-1db406ce9369 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:34.064579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a420df45-7697-4078-82ea-1db406ce9369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:34.132447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1253/2527] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:34.142027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bee59-c46c-4672-a1a2-7b05888b8184 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.142299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bee59-c46c-4672-a1a2-7b05888b8184 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.142563 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bee59-c46c-4672-a1a2-7b05888b8184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.163743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-796bee59-c46c-4672-a1a2-7b05888b8184 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:34.165252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-796bee59-c46c-4672-a1a2-7b05888b8184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:34.176112 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1275/2528] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:34.180462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420df45-7697-4078-82ea-1db406ce9369 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.190733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420df45-7697-4078-82ea-1db406ce9369 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.196289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.196887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.197157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.197417 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.198060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:34.198355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.200048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.212237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1254/2529] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:57:34.216944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bee59-c46c-4672-a1a2-7b05888b8184 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.226493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bee59-c46c-4672-a1a2-7b05888b8184 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.233169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.233768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.234043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.234310 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.234926 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.236881 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.272254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1276/2530] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:34 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:34.277673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.293212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663fc00-6976-4e0e-9b52-9d029f728dc0 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.352887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.353486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.353752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.354023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.354661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.356656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.379412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1255/2531] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:34 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:57:34.383787 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.393757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22f7f5-a23b-47d5-9d3a-5f46417f4dac admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.398208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.399100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.399366 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.399624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.400280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.401880 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.411766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1277/2532] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:34.415593 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.426699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2019-d2c0-45bb-ad07-62745b417a18 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.432660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f3a63-ddcf-4f1d-867e-600380bf6626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.433288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f3a63-ddcf-4f1d-867e-600380bf6626 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.433719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f3a63-ddcf-4f1d-867e-600380bf6626 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.433996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f3a63-ddcf-4f1d-867e-600380bf6626 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.434636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f3a63-ddcf-4f1d-867e-600380bf6626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.436249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f3a63-ddcf-4f1d-867e-600380bf6626 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.447444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1256/2533] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:34.452536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.463441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd0b9-03a1-4f13-9348-193d9c378e69 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.469797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.470450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.470739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.470977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] PATH_INFO: `/v3/projects/cd7688805d604c24a4a49e65e6cac30f/users/8bd135a735d244b5bddbfb58b568e68d/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.486803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7688805d604c24a4a49e65e6cac30f, user_id=8bd135a735d244b5bddbfb58b568e68d, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.489516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.504531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1278/2534] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:34 2022] PUT /identity/v3/projects/cd7688805d604c24a4a49e65e6cac30f/users/8bd135a735d244b5bddbfb58b568e68d/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:57:34.516273 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc4875-70b7-4cd5-816c-c078cd654ae4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.516581 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc4875-70b7-4cd5-816c-c078cd654ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.516850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc4875-70b7-4cd5-816c-c078cd654ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.546295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-23dc4875-70b7-4cd5-816c-c078cd654ae4 None None] MFA Rules not processed for user `8bd135a735d244b5bddbfb58b568e68d`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:34.547502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-23dc4875-70b7-4cd5-816c-c078cd654ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:34.605687 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:34.620349 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1257/2535] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:34.625356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.636184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:34.699214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2b4f-c8ea-48c6-8188-7919aef9da2d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.704188 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.704854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.705157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.705597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.706254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.708219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.737524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1279/2536] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:34 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:34.741752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc4875-70b7-4cd5-816c-c078cd654ae4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.753656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc4875-70b7-4cd5-816c-c078cd654ae4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.814315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.814921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.815191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.815454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.816070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.818065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.842274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1258/2537] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:34 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:34.846487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.856905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea29606-29fb-445e-b39f-50d6cdcb3939 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.861468 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.862096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.862422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.862824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.863618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.865255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.876109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1280/2538] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:57:34.880529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.891041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dfd97-27a3-43f8-a442-6ec786a76717 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:34.895942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:34.896631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:34.896911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:34.897175 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] PATH_INFO: `/v3/projects/cc21a851c74e46efb69289f4d6e9e461/users/b120b62f468549f79fe3760311f7651c/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:34.914154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc21a851c74e46efb69289f4d6e9e461, user_id=b120b62f468549f79fe3760311f7651c, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:34.917393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:34.932638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1259/2539] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:34 2022] PUT /identity/v3/projects/cc21a851c74e46efb69289f4d6e9e461/users/b120b62f468549f79fe3760311f7651c/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:34.937023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:34.950074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20582ba-a8e4-4155-9853-3f6a9f129cd7 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:35.017321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dab8ea-8bbb-47e8-9989-d7680db89249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:35.017987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab8ea-8bbb-47e8-9989-d7680db89249 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:35.018262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab8ea-8bbb-47e8-9989-d7680db89249 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:35.018527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dab8ea-8bbb-47e8-9989-d7680db89249 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:35.019186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dab8ea-8bbb-47e8-9989-d7680db89249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:35.020861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dab8ea-8bbb-47e8-9989-d7680db89249 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:35.031320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1281/2540] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:35.035820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:35.047330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976e3d-8cc7-41ad-8a5f-94f46233004b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:35.052531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:35.053212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:35.053481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:35.053750 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] PATH_INFO: `/v3/projects/cc21a851c74e46efb69289f4d6e9e461/users/b120b62f468549f79fe3760311f7651c/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:35.055761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc21a851c74e46efb69289f4d6e9e461, user_id=b120b62f468549f79fe3760311f7651c, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:35.058615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:35.071998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1260/2541] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:35 2022] PUT /identity/v3/projects/cc21a851c74e46efb69289f4d6e9e461/users/b120b62f468549f79fe3760311f7651c/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:35.083388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f2bb4-ae79-4b5f-91c2-f645927e85c9 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:35.083721 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f2bb4-ae79-4b5f-91c2-f645927e85c9 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:35.084083 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f2bb4-ae79-4b5f-91c2-f645927e85c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:35.120517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5a0f2bb4-ae79-4b5f-91c2-f645927e85c9 None None] MFA Rules not processed for user `b120b62f468549f79fe3760311f7651c`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:35.121727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5a0f2bb4-ae79-4b5f-91c2-f645927e85c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:35.193649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1282/2542] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:35.204963 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:35.206797 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:35.266070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96d15f-a841-4f78-9dcc-ddad45b4ec40 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:35.270012 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882a75-fc03-42ec-b265-b6009e0540ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:35.270642 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882a75-fc03-42ec-b265-b6009e0540ca service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:35.270912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882a75-fc03-42ec-b265-b6009e0540ca service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:35.271186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882a75-fc03-42ec-b265-b6009e0540ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:35.282141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71882a75-fc03-42ec-b265-b6009e0540ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:35.283579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71882a75-fc03-42ec-b265-b6009e0540ca service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:35.294083 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1261/2543] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:36.761035 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f2bb4-ae79-4b5f-91c2-f645927e85c9 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:36.765933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:36.826931 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f2bb4-ae79-4b5f-91c2-f645927e85c9 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:36.831473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:36.832090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:36.832417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:36.832646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] PATH_INFO: `/v3/users/8bd135a735d244b5bddbfb58b568e68d` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:36.833770 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd135a735d244b5bddbfb58b568e68d)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:36.835742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:36.889150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd135a735d244b5bddbfb58b568e68d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:36.889995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd135a735d244b5bddbfb58b568e68d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:36.890597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd135a735d244b5bddbfb58b568e68d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:36.898335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd135a735d244b5bddbfb58b568e68d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:36.907854 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd135a735d244b5bddbfb58b568e68d'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:36.925560 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1283/2544] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:36 2022] DELETE /identity/v3/users/8bd135a735d244b5bddbfb58b568e68d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:36.930293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:36.930770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:36.930906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:36.949788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:36.958272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:37.004015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:37.023048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1262/2545] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:37.033626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:37.035147 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:37.095103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f3ecb-e450-4a65-bba9-693431287723 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:37.099068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:37.099688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:37.099962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:37.100228 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:37.110450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:37.112105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:37.122630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1284/2546] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:37.412805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:37.414302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:37.430438 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae158b5-2e5e-4594-b0b1-345cdd566a61 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:37.437079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:37.438102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:37.438102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:37.438766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] PATH_INFO: `/v3/projects/cd7688805d604c24a4a49e65e6cac30f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:37.440225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7688805d604c24a4a49e65e6cac30f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:37.442172 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:37.492152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7688805d604c24a4a49e65e6cac30f', 'request_id': 'req-c73d4708-3ab7-42b4-a87e-883826361f37'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.497919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7688805d604c24a4a49e65e6cac30f', 'request_id': 'req-c73d4708-3ab7-42b4-a87e-883826361f37'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.504237 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7688805d604c24a4a49e65e6cac30f', 'request_id': 'req-c73d4708-3ab7-42b4-a87e-883826361f37'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.517640 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] The token cache is being invalidate because project cd7688805d604c24a4a49e65e6cac30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:37.517924 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.518524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.520039 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1263/2547] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:37 2022] DELETE /identity/v3/projects/cd7688805d604c24a4a49e65e6cac30f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:37.524523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:37.538470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fb224-f4e3-4220-ba19-87ce27cea1e3 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:37.560348 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:37.604550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:37.605413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:37.605795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:37.606122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] PATH_INFO: `/v3/users/b120b62f468549f79fe3760311f7651c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:37.607298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b120b62f468549f79fe3760311f7651c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:37.609254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:37.659186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b120b62f468549f79fe3760311f7651c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.659851 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b120b62f468549f79fe3760311f7651c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.660492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b120b62f468549f79fe3760311f7651c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.668694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b120b62f468549f79fe3760311f7651c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.677335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a5d0d27b-4478-4d69-929f-e15063a661d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b120b62f468549f79fe3760311f7651c'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:37.695169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1285/2548] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:37 2022] DELETE /identity/v3/users/b120b62f468549f79fe3760311f7651c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:38.077832 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.079432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:38.094051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d4708-3ab7-42b4-a87e-883826361f37 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.159938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.160602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.160873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.161158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] PATH_INFO: `/v3/projects/cc21a851c74e46efb69289f4d6e9e461` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.162450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc21a851c74e46efb69289f4d6e9e461)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.164456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.220761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc21a851c74e46efb69289f4d6e9e461', 'request_id': 'req-18e7bc97-b147-417c-9f0b-c14d708602f3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:38.226166 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc21a851c74e46efb69289f4d6e9e461', 'request_id': 'req-18e7bc97-b147-417c-9f0b-c14d708602f3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:38.233904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc21a851c74e46efb69289f4d6e9e461', 'request_id': 'req-18e7bc97-b147-417c-9f0b-c14d708602f3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:38.251264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] The token cache is being invalidate because project cc21a851c74e46efb69289f4d6e9e461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:38.251612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:38.252355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-18e7bc97-b147-417c-9f0b-c14d708602f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:38.253937 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1264/2549] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:38 2022] DELETE /identity/v3/projects/cc21a851c74e46efb69289f4d6e9e461 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:38.259131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b206b-b5d6-477d-9f5a-7f114d8c50b5 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.259630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b206b-b5d6-477d-9f5a-7f114d8c50b5 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.259816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b206b-b5d6-477d-9f5a-7f114d8c50b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.281632 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-ef8b206b-b5d6-477d-9f5a-7f114d8c50b5 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:38.283034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-ef8b206b-b5d6-477d-9f5a-7f114d8c50b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:38.359022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1286/2550] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:38.366878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217c7b2-83f5-445c-aeac-5ee4612bc3c3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.367600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217c7b2-83f5-445c-aeac-5ee4612bc3c3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.367600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217c7b2-83f5-445c-aeac-5ee4612bc3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.379584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:38.392621 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3217c7b2-83f5-445c-aeac-5ee4612bc3c3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:38.395207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3217c7b2-83f5-445c-aeac-5ee4612bc3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:38.405042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1265/2551] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:38.409417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b206b-b5d6-477d-9f5a-7f114d8c50b5 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.420158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b206b-b5d6-477d-9f5a-7f114d8c50b5 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.426381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.426565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.426865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.427141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.428233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:38.428233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.429742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.446708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1287/2552] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:57:38.448134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217c7b2-83f5-445c-aeac-5ee4612bc3c3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.459983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217c7b2-83f5-445c-aeac-5ee4612bc3c3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.465516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.465733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.465961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.466218 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.467655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.468773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.500644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1266/2553] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:38 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:38.505167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.517703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e5412-2f5a-413e-b654-4e69f070563d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.549298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:38.581483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.582213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.582522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.582731 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.583351 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.585295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.609339 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1288/2554] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:38 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:57:38.613543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.624776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752e6e0-de48-4b07-88f9-1440fc8a9a13 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.629834 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.630540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.630890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.631241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.631982 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.633845 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.644921 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1267/2555] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:38.649538 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.660722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b4f70-d399-4af8-acd8-ee1772420bbe admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.665528 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6e4e7-a759-4af6-99fc-bcad7f1fc0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.666138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e4e7-a759-4af6-99fc-bcad7f1fc0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.666403 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e4e7-a759-4af6-99fc-bcad7f1fc0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.666666 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e4e7-a759-4af6-99fc-bcad7f1fc0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.667330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6e4e7-a759-4af6-99fc-bcad7f1fc0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.669021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6e4e7-a759-4af6-99fc-bcad7f1fc0d2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.680230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1289/2556] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:38.684862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.695850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4ac3-b92b-430f-bc1b-a46e1c80c4be admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.701525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.702194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.702465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.702747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] PATH_INFO: `/v3/projects/58104c15be594c7fa37494d4c266f59a/users/decc4784a6224a87b0d4e44cd8d6fd0f/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.719332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=58104c15be594c7fa37494d4c266f59a, user_id=decc4784a6224a87b0d4e44cd8d6fd0f, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.722070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.737414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1268/2557] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:38 2022] PUT /identity/v3/projects/58104c15be594c7fa37494d4c266f59a/users/decc4784a6224a87b0d4e44cd8d6fd0f/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:38.749444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544328a-b740-4c03-a3ad-4b790137b2c4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.749726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544328a-b740-4c03-a3ad-4b790137b2c4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.750001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544328a-b740-4c03-a3ad-4b790137b2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.781841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d544328a-b740-4c03-a3ad-4b790137b2c4 None None] MFA Rules not processed for user `decc4784a6224a87b0d4e44cd8d6fd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:38.783068 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d544328a-b740-4c03-a3ad-4b790137b2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:38.849077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1290/2558] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:38.853748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.918300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f566337-176d-4368-a015-ca8903d2f36c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:38.923158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:38.923752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:38.924019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:38.924271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:38.924910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:38.926820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:38.956749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1269/2559] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:38 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:38.961056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544328a-b740-4c03-a3ad-4b790137b2c4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:38.971812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544328a-b740-4c03-a3ad-4b790137b2c4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.028090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.028723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.028996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.029262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.029871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.031797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.057519 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1291/2560] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:38 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:39.062117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:39.073104 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a3163-dd9e-4040-9da2-42456437019b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.078154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.078721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.078999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.079261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.079901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.081565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.092194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1270/2561] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:39.096719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:39.107374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedf29a-66dc-427d-acba-6d317ae9310b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.113177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.113901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.114172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.114437 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] PATH_INFO: `/v3/projects/58a4db93511b437b8929af77366c55c8/users/8a94a1ccd19741a6a02f6443e6c29663/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.130222 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a4db93511b437b8929af77366c55c8, user_id=8a94a1ccd19741a6a02f6443e6c29663, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.132936 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.153494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1292/2562] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:39 2022] PUT /identity/v3/projects/58a4db93511b437b8929af77366c55c8/users/8a94a1ccd19741a6a02f6443e6c29663/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:39.157831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:39.169939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8317e0-9652-403b-9501-cdd187f5cb39 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.234873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed7794-29fc-4e58-b904-d27eadbf724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.235721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7794-29fc-4e58-b904-d27eadbf724d admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.236053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7794-29fc-4e58-b904-d27eadbf724d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.236422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed7794-29fc-4e58-b904-d27eadbf724d admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.237366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed7794-29fc-4e58-b904-d27eadbf724d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.239815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed7794-29fc-4e58-b904-d27eadbf724d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.252579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1271/2563] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:39.258359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:39.266020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:39.270596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a22621-41a1-4102-8411-7c08bd34662f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.277292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.278230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.278511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.278785 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] PATH_INFO: `/v3/projects/58a4db93511b437b8929af77366c55c8/users/8a94a1ccd19741a6a02f6443e6c29663/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.280855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a4db93511b437b8929af77366c55c8, user_id=8a94a1ccd19741a6a02f6443e6c29663, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.283530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.293715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1293/2564] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:39 2022] PUT /identity/v3/projects/58a4db93511b437b8929af77366c55c8/users/8a94a1ccd19741a6a02f6443e6c29663/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:39.304136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6168e4-a97f-427b-b76f-62c14340cad4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.304455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6168e4-a97f-427b-b76f-62c14340cad4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.304731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6168e4-a97f-427b-b76f-62c14340cad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.326168 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:39.334679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-fe6168e4-a97f-427b-b76f-62c14340cad4 None None] MFA Rules not processed for user `8a94a1ccd19741a6a02f6443e6c29663`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:39.336495 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fe6168e4-a97f-427b-b76f-62c14340cad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:39.406255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1272/2565] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:39.419742 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:39.421665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:39.478432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f278db-74fb-4639-9a7b-e5bce3a4833e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.482567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6bae-a0f0-4491-ae5d-beefb5680760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.483162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6bae-a0f0-4491-ae5d-beefb5680760 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.483431 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6bae-a0f0-4491-ae5d-beefb5680760 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.483694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6bae-a0f0-4491-ae5d-beefb5680760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.492826 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6bae-a0f0-4491-ae5d-beefb5680760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.494255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6bae-a0f0-4491-ae5d-beefb5680760 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.503750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1294/2566] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:39.533174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6168e4-a97f-427b-b76f-62c14340cad4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:39.596655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6168e4-a97f-427b-b76f-62c14340cad4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.601484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.602107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.602382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.602650 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] PATH_INFO: `/v3/users/decc4784a6224a87b0d4e44cd8d6fd0f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.603830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=decc4784a6224a87b0d4e44cd8d6fd0f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.607435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.657674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'decc4784a6224a87b0d4e44cd8d6fd0f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:39.664711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'decc4784a6224a87b0d4e44cd8d6fd0f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:39.665595 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'decc4784a6224a87b0d4e44cd8d6fd0f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:39.673765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'decc4784a6224a87b0d4e44cd8d6fd0f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:39.685638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'decc4784a6224a87b0d4e44cd8d6fd0f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:39.692005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1273/2567] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:39 2022] DELETE /identity/v3/users/decc4784a6224a87b0d4e44cd8d6fd0f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:39.696715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.696988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.697253 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.717022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:39.718215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:39.778727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1295/2568] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:39.789102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:39.790699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:39.810294 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:39.851882 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad485a-4ccc-4187-b50b-aa7e561ce870 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:39.855661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:39.856295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:39.856561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:39.856833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:39.866041 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:39.867480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:39.877248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1274/2569] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:40.157138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:40.158805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:40.173416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ddb8-6bfe-4070-a391-ea91d8d06da7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:40.178174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:40.178811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:40.179247 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:40.179534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] PATH_INFO: `/v3/projects/58104c15be594c7fa37494d4c266f59a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:40.180708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=58104c15be594c7fa37494d4c266f59a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:40.182640 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:40.234167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58104c15be594c7fa37494d4c266f59a', 'request_id': 'req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.240933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58104c15be594c7fa37494d4c266f59a', 'request_id': 'req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.247410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58104c15be594c7fa37494d4c266f59a', 'request_id': 'req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.262722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] The token cache is being invalidate because project 58104c15be594c7fa37494d4c266f59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:40.262995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.263760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.265272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1296/2570] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:40 2022] DELETE /identity/v3/projects/58104c15be594c7fa37494d4c266f59a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:40.269617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:40.280170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d2255-887f-419f-9ce0-c384e91b83e0 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:40.339240 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:40.339598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:40.339884 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:40.340148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] PATH_INFO: `/v3/users/8a94a1ccd19741a6a02f6443e6c29663` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:40.341557 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a94a1ccd19741a6a02f6443e6c29663)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:40.343422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:40.397822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a94a1ccd19741a6a02f6443e6c29663'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.404978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a94a1ccd19741a6a02f6443e6c29663'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.405638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a94a1ccd19741a6a02f6443e6c29663'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.413943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a94a1ccd19741a6a02f6443e6c29663'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.426193 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-c4702757-6201-493c-bc28-be73dd2738f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a94a1ccd19741a6a02f6443e6c29663'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.432942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1275/2571] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:40 2022] DELETE /identity/v3/users/8a94a1ccd19741a6a02f6443e6c29663 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:40.719814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:40.721393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:40.735260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac66b5b-a154-4b15-ac78-f14fd14b6c5b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:40.748982 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:40.797964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:40.798486 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:40.798762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:40.799005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] PATH_INFO: `/v3/projects/58a4db93511b437b8929af77366c55c8` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:40.800203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a4db93511b437b8929af77366c55c8)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:40.802235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:40.852886 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a4db93511b437b8929af77366c55c8', 'request_id': 'req-7b2076a1-526b-4411-9d19-1b8d94f45197'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.859368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a4db93511b437b8929af77366c55c8', 'request_id': 'req-7b2076a1-526b-4411-9d19-1b8d94f45197'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.864870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a4db93511b437b8929af77366c55c8', 'request_id': 'req-7b2076a1-526b-4411-9d19-1b8d94f45197'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.879463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] The token cache is being invalidate because project 58a4db93511b437b8929af77366c55c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:40.879748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.880513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-7b2076a1-526b-4411-9d19-1b8d94f45197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:40.882082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1297/2572] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:40 2022] DELETE /identity/v3/projects/58a4db93511b437b8929af77366c55c8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:40.887101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dc2a3-59f1-428a-b8ec-d558c4d15829 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:40.887385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dc2a3-59f1-428a-b8ec-d558c4d15829 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:40.887658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dc2a3-59f1-428a-b8ec-d558c4d15829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:40.908720 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-f67dc2a3-59f1-428a-b8ec-d558c4d15829 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:40.910030 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-f67dc2a3-59f1-428a-b8ec-d558c4d15829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:40.944271 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:40.979254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1276/2573] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:57:40.987776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae5e67-7c68-4db4-9fa7-de5f82e523ed None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:40.988082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae5e67-7c68-4db4-9fa7-de5f82e523ed None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:40.988376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae5e67-7c68-4db4-9fa7-de5f82e523ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.008625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-61ae5e67-7c68-4db4-9fa7-de5f82e523ed None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:41.009833 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-61ae5e67-7c68-4db4-9fa7-de5f82e523ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:41.018189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1298/2574] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:41.022712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dc2a3-59f1-428a-b8ec-d558c4d15829 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.033357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dc2a3-59f1-428a-b8ec-d558c4d15829 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.038413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.039005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.039275 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.039538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.040186 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:41.040513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.042121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.053796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1277/2575] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:57:41.058548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae5e67-7c68-4db4-9fa7-de5f82e523ed None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.069017 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae5e67-7c68-4db4-9fa7-de5f82e523ed None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.074630 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.075218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.075489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.075855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.077171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.079063 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.109469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1299/2576] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:41 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:41.113894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.124026 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f563d-2e4e-4bd3-aa82-db83d6a04ee8 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.181605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.182213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.182491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.182759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.183377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.185386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.208435 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1278/2577] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:41 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:41.212764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.225557 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f9ffd-6777-49b9-9dea-f23c679da75a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.230342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.231200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.231474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.231740 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.232665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.234051 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.244189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1300/2578] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:41.248605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.260236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cfc8c-5f59-4621-972c-e274023cf267 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.265220 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877c874-873d-4d2d-9ea1-44ee41eec54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.266337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c874-873d-4d2d-9ea1-44ee41eec54c admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.266337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c874-873d-4d2d-9ea1-44ee41eec54c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.266648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c874-873d-4d2d-9ea1-44ee41eec54c admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.267389 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877c874-873d-4d2d-9ea1-44ee41eec54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.269085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877c874-873d-4d2d-9ea1-44ee41eec54c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.281615 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1279/2579] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:57:41.286364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.297801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b69d-754d-4a5c-97a9-39eeb3044769 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.302797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.303549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.303876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.304131 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] PATH_INFO: `/v3/projects/0ecd4b2bcd484effa55eb0eb6cfe3b83/users/b5ae494035104042920185cb72604671/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.321272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecd4b2bcd484effa55eb0eb6cfe3b83, user_id=b5ae494035104042920185cb72604671, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.324197 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.342822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1301/2580] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:41 2022] PUT /identity/v3/projects/0ecd4b2bcd484effa55eb0eb6cfe3b83/users/b5ae494035104042920185cb72604671/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:41.354454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c27c47-6f8b-4b07-8f27-0472e264c06b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.354736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c27c47-6f8b-4b07-8f27-0472e264c06b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.355007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c27c47-6f8b-4b07-8f27-0472e264c06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.387327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-60c27c47-6f8b-4b07-8f27-0472e264c06b None None] MFA Rules not processed for user `b5ae494035104042920185cb72604671`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:41.389059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-60c27c47-6f8b-4b07-8f27-0472e264c06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:41.455994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1280/2581] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:41.468661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.470433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:41.520633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:41.533752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7892d28-7112-40cb-83be-651fd43a794d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.538021 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9d6b1-0b7f-484e-9890-4a1ebf14ed0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.538784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d6b1-0b7f-484e-9890-4a1ebf14ed0d service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.539263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d6b1-0b7f-484e-9890-4a1ebf14ed0d service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.539592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d6b1-0b7f-484e-9890-4a1ebf14ed0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.549497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9d6b1-0b7f-484e-9890-4a1ebf14ed0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.550992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9d6b1-0b7f-484e-9890-4a1ebf14ed0d service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.560151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1302/2582] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:41.582316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c27c47-6f8b-4b07-8f27-0472e264c06b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.621033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:41.656588 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c27c47-6f8b-4b07-8f27-0472e264c06b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.661959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.662592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.662863 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.663127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] PATH_INFO: `/v3/users/b5ae494035104042920185cb72604671` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.664423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ae494035104042920185cb72604671)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.666388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.720080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ae494035104042920185cb72604671'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:41.727870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ae494035104042920185cb72604671'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:41.728727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ae494035104042920185cb72604671'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:41.738972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ae494035104042920185cb72604671'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:41.752893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ae494035104042920185cb72604671'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:41.759052 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1281/2583] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:41 2022] DELETE /identity/v3/users/b5ae494035104042920185cb72604671 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:41.764415 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.764941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.764941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.788734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:41.790518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:41.851904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1303/2584] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:41.862057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:41.863847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:41.926531 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d3cc5-da10-4aa9-b053-091bcf6fbd70 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:41.931053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0743cc2-41f8-4da5-9d91-67ceb517a0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:41.931651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0743cc2-41f8-4da5-9d91-67ceb517a0ef service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:41.931931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0743cc2-41f8-4da5-9d91-67ceb517a0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:41.932192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0743cc2-41f8-4da5-9d91-67ceb517a0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:41.943395 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0743cc2-41f8-4da5-9d91-67ceb517a0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:41.944887 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0743cc2-41f8-4da5-9d91-67ceb517a0ef service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:41.954749 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1282/2585] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:42.234245 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.235796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:42.248762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1145e1a-4427-4db7-a97e-174ddcfb1f58 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:42.254056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:42.254701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.254944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.255213 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] PATH_INFO: `/v3/projects/0ecd4b2bcd484effa55eb0eb6cfe3b83` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.256422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ecd4b2bcd484effa55eb0eb6cfe3b83)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:42.258491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:42.307981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ecd4b2bcd484effa55eb0eb6cfe3b83', 'request_id': 'req-63a88d97-570d-4605-87c3-f0406a759219'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:42.314149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ecd4b2bcd484effa55eb0eb6cfe3b83', 'request_id': 'req-63a88d97-570d-4605-87c3-f0406a759219'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:42.320283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ecd4b2bcd484effa55eb0eb6cfe3b83', 'request_id': 'req-63a88d97-570d-4605-87c3-f0406a759219'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:42.334504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] The token cache is being invalidate because project 0ecd4b2bcd484effa55eb0eb6cfe3b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:42.334788 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:42.335409 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-63a88d97-570d-4605-87c3-f0406a759219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:42.337175 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1304/2586] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:42 2022] DELETE /identity/v3/projects/0ecd4b2bcd484effa55eb0eb6cfe3b83 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:42.343233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d57cb-9d31-464f-ba5f-e7f254f7aee0 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.343754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d57cb-9d31-464f-ba5f-e7f254f7aee0 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.343754 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d57cb-9d31-464f-ba5f-e7f254f7aee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.363952 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-209d57cb-9d31-464f-ba5f-e7f254f7aee0 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:42.365123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-209d57cb-9d31-464f-ba5f-e7f254f7aee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:42.416010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:42.429010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1283/2587] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:42.437207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975576ca-cfb4-4396-af50-7f0c891f9847 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.437482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975576ca-cfb4-4396-af50-7f0c891f9847 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.437746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975576ca-cfb4-4396-af50-7f0c891f9847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.458705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-975576ca-cfb4-4396-af50-7f0c891f9847 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:42.459899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-975576ca-cfb4-4396-af50-7f0c891f9847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:42.468981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1305/2588] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:42.473539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d57cb-9d31-464f-ba5f-e7f254f7aee0 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.484617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d57cb-9d31-464f-ba5f-e7f254f7aee0 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:42.493174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:42.494020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.494293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.494567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.495493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:42.495866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:42.498234 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:42.515236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1284/2589] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:57:42.519932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975576ca-cfb4-4396-af50-7f0c891f9847 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.530193 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975576ca-cfb4-4396-af50-7f0c891f9847 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:42.535867 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:42.536509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.536782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.537046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.537661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:42.539570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:42.572408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1306/2590] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:42 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:42.577048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.588298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e469c-3f4a-4429-9ec1-19f888b6e332 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:42.648171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:42.649180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.649454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.649712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.650339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:42.652269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:42.677159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1285/2591] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:42 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:42.681975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.693330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e14ea-08cb-4e4e-8f54-f48f9f1ca3bf admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:42.698108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:42.698783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.699115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.699441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.700155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:42.702007 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:42.713401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1307/2592] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:42.717752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.728022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3703ad-9b2f-4ad9-890c-a05f22dd4ff7 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:42.736998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6b51c-7c6e-4c2d-bb56-dc821e384e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:42.737604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6b51c-7c6e-4c2d-bb56-dc821e384e30 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.737864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6b51c-7c6e-4c2d-bb56-dc821e384e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.738136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6b51c-7c6e-4c2d-bb56-dc821e384e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.738778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6b51c-7c6e-4c2d-bb56-dc821e384e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:42.740387 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6b51c-7c6e-4c2d-bb56-dc821e384e30 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:42.753451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1286/2593] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:42.758750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.769186 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65987a-41d8-4972-93d7-e1033e30f513 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:42.774238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:42.774877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.775153 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.775422 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] PATH_INFO: `/v3/projects/c629d3fc3d894e83b727f3a90e41d458/users/438f4732ed444458bc324635129aae68/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.790861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c629d3fc3d894e83b727f3a90e41d458, user_id=438f4732ed444458bc324635129aae68, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:42.793597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:42.808562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1308/2594] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:42 2022] PUT /identity/v3/projects/c629d3fc3d894e83b727f3a90e41d458/users/438f4732ed444458bc324635129aae68/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:42.819405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de2b5a-7ce2-4842-a71f-87bfc21d9c7d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:42.819689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de2b5a-7ce2-4842-a71f-87bfc21d9c7d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:42.819962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de2b5a-7ce2-4842-a71f-87bfc21d9c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:42.850636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-25de2b5a-7ce2-4842-a71f-87bfc21d9c7d None None] MFA Rules not processed for user `438f4732ed444458bc324635129aae68`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:42.851896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-25de2b5a-7ce2-4842-a71f-87bfc21d9c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:42.918214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1287/2595] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:42.923633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:42.937169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:42.993684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac76821-ebce-4a72-a53e-7395ada8ba8f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:43.001217 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:43.002209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.002505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.002849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.003719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:43.006847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:43.045855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1309/2596] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:42 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:43.049973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de2b5a-7ce2-4842-a71f-87bfc21d9c7d None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:43.061051 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de2b5a-7ce2-4842-a71f-87bfc21d9c7d None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:43.107959 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:43.119678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:43.120444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.120719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.120984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.121627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:43.123542 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:43.147191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1288/2597] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:43 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:43.151572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:43.163321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619c7f-ef29-4dbf-91d9-bda241bb1a6e admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:43.169183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:43.169757 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.170032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.170294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.170944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:43.173303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:43.184992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1310/2598] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:43.190048 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:43.200670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c33a6-e8a0-462b-bec2-b24ef359eecd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:43.206038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:43.206933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.207211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.207483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] PATH_INFO: `/v3/projects/4a86babcb73448a08bf7fa0077640cf5/users/ec1934072da34621a809ef731be741a4/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.225091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a86babcb73448a08bf7fa0077640cf5, user_id=ec1934072da34621a809ef731be741a4, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:43.227999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:43.243189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1289/2599] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:43 2022] PUT /identity/v3/projects/4a86babcb73448a08bf7fa0077640cf5/users/ec1934072da34621a809ef731be741a4/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:43.247322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:43.258263 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39899a89-8c2b-46ad-8dc9-ade534078c8a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:43.323688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd06625-bac9-4f8e-9ffa-c7736cebf8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:43.324346 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd06625-bac9-4f8e-9ffa-c7736cebf8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.324620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd06625-bac9-4f8e-9ffa-c7736cebf8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.324892 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd06625-bac9-4f8e-9ffa-c7736cebf8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.325535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd06625-bac9-4f8e-9ffa-c7736cebf8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:43.327155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd06625-bac9-4f8e-9ffa-c7736cebf8b9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:43.341493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1311/2600] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:43.346528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:43.357112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df42e71-7b65-450a-9915-3be0cfb0968d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:43.362285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:43.362849 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.363121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.363392 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] PATH_INFO: `/v3/projects/4a86babcb73448a08bf7fa0077640cf5/users/ec1934072da34621a809ef731be741a4/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.365371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a86babcb73448a08bf7fa0077640cf5, user_id=ec1934072da34621a809ef731be741a4, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:43.368040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:43.378630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1290/2601] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:43 2022] PUT /identity/v3/projects/4a86babcb73448a08bf7fa0077640cf5/users/ec1934072da34621a809ef731be741a4/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:43.450397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a202b2-f064-47b2-afc3-c2349c27b000 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.450599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a202b2-f064-47b2-afc3-c2349c27b000 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.450839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a202b2-f064-47b2-afc3-c2349c27b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.479935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-a0a202b2-f064-47b2-afc3-c2349c27b000 None None] MFA Rules not processed for user `ec1934072da34621a809ef731be741a4`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:43.481178 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a0a202b2-f064-47b2-afc3-c2349c27b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:43.554550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1312/2602] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:43.565951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:43.567740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:43.628756 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294efd-613f-4a36-9d6d-197996572941 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:43.632637 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec48620-d420-4379-b9d9-05a418574b5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:43.633241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec48620-d420-4379-b9d9-05a418574b5c service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:43.633517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec48620-d420-4379-b9d9-05a418574b5c service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:43.633807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec48620-d420-4379-b9d9-05a418574b5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:43.691419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec48620-d420-4379-b9d9-05a418574b5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:43.692918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec48620-d420-4379-b9d9-05a418574b5c service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:43.702992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1291/2603] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:46.366974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a202b2-f064-47b2-afc3-c2349c27b000 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:46.371974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:46.434453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a202b2-f064-47b2-afc3-c2349c27b000 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:46.439709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:46.440304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:46.440591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:46.440857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] PATH_INFO: `/v3/users/438f4732ed444458bc324635129aae68` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:46.441998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438f4732ed444458bc324635129aae68)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:46.443920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:46.493363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438f4732ed444458bc324635129aae68'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:46.494254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438f4732ed444458bc324635129aae68'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:46.494887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438f4732ed444458bc324635129aae68'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:46.502124 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438f4732ed444458bc324635129aae68'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:46.511296 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438f4732ed444458bc324635129aae68'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:46.531123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1313/2604] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:46 2022] DELETE /identity/v3/users/438f4732ed444458bc324635129aae68 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:46.536108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:46.536682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:46.536682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:46.556136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:46.557444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:46.610375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:46.624145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1292/2605] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:46.630975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:46.632473 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:46.694060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e6215-0a0b-4699-8cf1-0960514b9ff9 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:46.698520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:46.699114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:46.699382 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:46.699615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:46.709695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:46.711114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:46.721001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1314/2606] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:47.014433 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:47.015942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:47.028776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7477eea-6449-4d5e-a556-71b14cdeaf73 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:47.034257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:47.034897 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:47.035185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:47.035445 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] PATH_INFO: `/v3/projects/c629d3fc3d894e83b727f3a90e41d458` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:47.037381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c629d3fc3d894e83b727f3a90e41d458)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:47.038747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:47.087488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c629d3fc3d894e83b727f3a90e41d458', 'request_id': 'req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.092862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c629d3fc3d894e83b727f3a90e41d458', 'request_id': 'req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.098994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c629d3fc3d894e83b727f3a90e41d458', 'request_id': 'req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.116586 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] The token cache is being invalidate because project c629d3fc3d894e83b727f3a90e41d458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:47.116909 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.117801 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.119752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1293/2607] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:47 2022] DELETE /identity/v3/projects/c629d3fc3d894e83b727f3a90e41d458 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:47.124164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:47.134825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c40a4d-10b0-4766-848f-0f59be25f25c service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:47.155840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:47.203338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:47.203965 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:47.204236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:47.204530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] PATH_INFO: `/v3/users/ec1934072da34621a809ef731be741a4` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:47.205704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1934072da34621a809ef731be741a4)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:47.207629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:47.263600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1934072da34621a809ef731be741a4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.264249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1934072da34621a809ef731be741a4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.264928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1934072da34621a809ef731be741a4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.272544 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1934072da34621a809ef731be741a4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.281558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28960083-5381-4991-97c8-d10e45156eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1934072da34621a809ef731be741a4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.304325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1315/2608] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:47 2022] DELETE /identity/v3/users/ec1934072da34621a809ef731be741a4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:47.614264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:47.615777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:47.629375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177e2e-f593-4a5f-b825-9d2fca51a9b3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:47.688180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:47.688753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:47.689020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:47.689297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] PATH_INFO: `/v3/projects/4a86babcb73448a08bf7fa0077640cf5` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:47.690437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a86babcb73448a08bf7fa0077640cf5)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:47.692394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:47.738260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a86babcb73448a08bf7fa0077640cf5', 'request_id': 'req-2dabd96d-ce5a-4918-bfd4-e44743cb9072'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.743666 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a86babcb73448a08bf7fa0077640cf5', 'request_id': 'req-2dabd96d-ce5a-4918-bfd4-e44743cb9072'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.749703 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a86babcb73448a08bf7fa0077640cf5', 'request_id': 'req-2dabd96d-ce5a-4918-bfd4-e44743cb9072'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.762880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] The token cache is being invalidate because project 4a86babcb73448a08bf7fa0077640cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:47.763191 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.763826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2dabd96d-ce5a-4918-bfd4-e44743cb9072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:47.765367 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1294/2609] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:47 2022] DELETE /identity/v3/projects/4a86babcb73448a08bf7fa0077640cf5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:47.771207 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fefed9-a2fc-43a5-b4f6-bed35a27aa05 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:47.771708 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fefed9-a2fc-43a5-b4f6-bed35a27aa05 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:47.771825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fefed9-a2fc-43a5-b4f6-bed35a27aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:47.792241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-f5fefed9-a2fc-43a5-b4f6-bed35a27aa05 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:47.793462 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-f5fefed9-a2fc-43a5-b4f6-bed35a27aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:47.856328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1316/2610] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:47.864378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2340ee-0476-4a56-8b3e-4180a9c94f94 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:47.864632 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2340ee-0476-4a56-8b3e-4180a9c94f94 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:47.864946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2340ee-0476-4a56-8b3e-4180a9c94f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:47.886535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-5d2340ee-0476-4a56-8b3e-4180a9c94f94 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:47.887742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-5d2340ee-0476-4a56-8b3e-4180a9c94f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:47.897249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1295/2611] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:57:47.906719 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fefed9-a2fc-43a5-b4f6-bed35a27aa05 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:47.917428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fefed9-a2fc-43a5-b4f6-bed35a27aa05 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:47.922456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:47.923050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:47.923319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:47.923580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:47.924199 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:47.924525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:47.926120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:47.939340 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1317/2612] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:57:47.942562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2340ee-0476-4a56-8b3e-4180a9c94f94 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:47.948724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:47.952699 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2340ee-0476-4a56-8b3e-4180a9c94f94 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:47.960300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:47.961414 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:47.961790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:47.962362 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:47.963449 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:47.966223 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:48.005102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1296/2613] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:47 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:48.009503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:48.020570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11489d4-6de1-4fbc-a172-f48734f9a8f2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:48.044504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:48.046229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:48.050906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:48.084392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:48.085150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:48.085525 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:48.085800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:48.086430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:48.088598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:48.106944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5b9af-ca51-4a74-92d4-9458f7a10306 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:48.110534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:48.114274 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1318/2614] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:48 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:48.114422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:48.114422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:48.114422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:48.114422 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:48.115505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:48.125242 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d2be-2256-4640-a80e-294f9375879a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:48.125811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:48.127348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:48.129970 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:48.130558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:48.130832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:48.131097 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:48.131718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:48.133364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:48.136936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1297/2615] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 10:57:48.146540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1319/2616] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:57:48.147820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:48.157746 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fea957-0523-480a-bb25-996455c1121e service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:48.162401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c6720-431d-4953-85cc-eadeb877ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:48.162992 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6720-431d-4953-85cc-eadeb877ea3f admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:48.163264 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6720-431d-4953-85cc-eadeb877ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:48.163527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6720-431d-4953-85cc-eadeb877ea3f admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:48.164154 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c6720-431d-4953-85cc-eadeb877ea3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:48.165825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c6720-431d-4953-85cc-eadeb877ea3f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:48.181314 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1298/2617] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:48.182780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:48.192733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f3918-00bb-4a0b-a13e-61c3704bc0fe admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:48.197397 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:48.198098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:48.198365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:48.198651 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] PATH_INFO: `/v3/projects/fafc312df1404fc6bbc28415bb744411/users/f16bdb81c5a249bb9f95e390dbd0dbfd/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:48.213261 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fafc312df1404fc6bbc28415bb744411, user_id=f16bdb81c5a249bb9f95e390dbd0dbfd, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:48.216040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:48.229699 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1320/2618] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:48 2022] PUT /identity/v3/projects/fafc312df1404fc6bbc28415bb744411/users/f16bdb81c5a249bb9f95e390dbd0dbfd/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:48.235046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:48.235332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:48.235986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:48.256856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:48.258067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:48.319352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1299/2619] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:57:48.333871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:48.392723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803e42-ad0e-4c80-b445-ff73514fbc24 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:48.397054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:48.397688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:48.398050 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:48.398318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:48.407928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:48.409496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:48.419882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1321/2620] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:51.734604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:51.736332 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:51.860296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834ea27-9fc8-4b16-988b-4e5c68eeddea None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:51.870420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:51.871959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:51.879419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:51.880830 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:51.881202 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:51.881782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:51.885106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:51.890005 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025d2a3-3891-465e-b214-111da8b8c7e3 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:51.895522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:51.896224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:51.896536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:51.896810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] PATH_INFO: `/v3/users/c0711a144b4c4432ab849367d3bd2b33` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:51.898161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0711a144b4c4432ab849367d3bd2b33)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:51.900777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:51.904779 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:51.910310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:51.930150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1300/2621] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:51.968643 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0711a144b4c4432ab849367d3bd2b33'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:51.969906 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0711a144b4c4432ab849367d3bd2b33'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:51.970581 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0711a144b4c4432ab849367d3bd2b33'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:51.979547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0711a144b4c4432ab849367d3bd2b33'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:51.989420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-ca813128-a745-4853-8453-745b755ad1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0711a144b4c4432ab849367d3bd2b33'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:52.007944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:52.013755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1322/2622] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:57:51 2022] DELETE /identity/v3/users/c0711a144b4c4432ab849367d3bd2b33 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:52.237622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:52.244644 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:52.250501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea068b-5e40-48b5-a03e-aa90c63d6604 service placement] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:52.329020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:52.329913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.330827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.330827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] PATH_INFO: `/v3/projects/1d3cc44ffed94a428ab9e49ebb3854e0` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.332155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3cc44ffed94a428ab9e49ebb3854e0)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:52.337037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:52.405031 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3cc44ffed94a428ab9e49ebb3854e0', 'request_id': 'req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:52.412016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3cc44ffed94a428ab9e49ebb3854e0', 'request_id': 'req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:52.420492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3cc44ffed94a428ab9e49ebb3854e0', 'request_id': 'req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:52.456567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] The token cache is being invalidate because project 1d3cc44ffed94a428ab9e49ebb3854e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:57:52.456567 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:52.457262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7fb0f0a2-12e2-4e5a-9cec-b1b41168415d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:57:52.468310 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1301/2623] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:57:52 2022] DELETE /identity/v3/projects/1d3cc44ffed94a428ab9e49ebb3854e0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:57:52.469985 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd55e9-4f7a-4a6b-95e9-e233a4ac4803 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.470267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd55e9-4f7a-4a6b-95e9-e233a4ac4803 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.470628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd55e9-4f7a-4a6b-95e9-e233a4ac4803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.499289 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-12fd55e9-4f7a-4a6b-95e9-e233a4ac4803 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:52.500637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-12fd55e9-4f7a-4a6b-95e9-e233a4ac4803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:52.585417 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1323/2624] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:52.594874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441536-fbac-448e-8f89-dd6ab7d4b1ac None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.595194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441536-fbac-448e-8f89-dd6ab7d4b1ac None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.595419 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441536-fbac-448e-8f89-dd6ab7d4b1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.624791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7c441536-fbac-448e-8f89-dd6ab7d4b1ac None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:52.626292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7c441536-fbac-448e-8f89-dd6ab7d4b1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:52.639354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1302/2625] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:52.641113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd55e9-4f7a-4a6b-95e9-e233a4ac4803 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:52.655459 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd55e9-4f7a-4a6b-95e9-e233a4ac4803 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:52.667767 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:52.668502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.668818 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.669085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.669739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:57:52.670032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:52.671652 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:52.695433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1324/2626] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 10:57:52.696134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c441536-fbac-448e-8f89-dd6ab7d4b1ac None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:52.710628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c441536-fbac-448e-8f89-dd6ab7d4b1ac None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:52.715988 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:52.716859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.717281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.717723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.719295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:52.721822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:52.760423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1303/2627] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:52 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:57:52.762810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:52.774998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d4b4d-645d-45e8-b12b-f060c251dc03 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:52.859531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:52.860347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.860874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.861355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.862478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:52.864735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:52.895773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1325/2628] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:52 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:52.900114 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:52.913348 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256826d3-5977-404e-a9c1-9271689bbbdb admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:52.919281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:52.920065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.920490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.920854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.921961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:52.924156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:52.936122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1304/2629] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:57:52.939840 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:52.950314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169c262-7dcd-4e54-9a09-849f053debf4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:52.955438 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd537fc1-38f3-4ad5-8812-04370f0b29f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:52.956160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537fc1-38f3-4ad5-8812-04370f0b29f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.956720 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537fc1-38f3-4ad5-8812-04370f0b29f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.956927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537fc1-38f3-4ad5-8812-04370f0b29f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:52.957741 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd537fc1-38f3-4ad5-8812-04370f0b29f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:52.959380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd537fc1-38f3-4ad5-8812-04370f0b29f3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:52.971137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1326/2630] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:52.975966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:52.987049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61b1ee-d6d6-4ec0-b38d-c2b2c9a3c0a9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:52.995200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:52.996698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:52.996976 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:52.997263 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] PATH_INFO: `/v3/projects/a952ae3bd5e14e39be82baa02aecf77d/users/45f446d513e94ccba51391c7d6b03876/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.015291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a952ae3bd5e14e39be82baa02aecf77d, user_id=45f446d513e94ccba51391c7d6b03876, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:53.019143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:53.043730 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1305/2631] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:52 2022] PUT /identity/v3/projects/a952ae3bd5e14e39be82baa02aecf77d/users/45f446d513e94ccba51391c7d6b03876/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:57:53.054402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceed2fc-ff0a-4fa4-a949-210cb9bf8aab None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.054800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceed2fc-ff0a-4fa4-a949-210cb9bf8aab None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.055162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceed2fc-ff0a-4fa4-a949-210cb9bf8aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.088011 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7ceed2fc-ff0a-4fa4-a949-210cb9bf8aab None None] MFA Rules not processed for user `45f446d513e94ccba51391c7d6b03876`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:53.089655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7ceed2fc-ff0a-4fa4-a949-210cb9bf8aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:53.208824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:53.244688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1327/2632] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:53.255820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:53.257676 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:53.331230 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:53.377919 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd16f-6f89-49e6-9cc3-486cb2362482 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.378244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd16f-6f89-49e6-9cc3-486cb2362482 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.378489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd16f-6f89-49e6-9cc3-486cb2362482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.401221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63a92f-78ca-4c66-b4f0-be16d7e934d1 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:53.407843 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:53.409436 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.409710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.409980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.412284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-deecd16f-6f89-49e6-9cc3-486cb2362482 None None] MFA Rules not processed for user `f16bdb81c5a249bb9f95e390dbd0dbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:53.413633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-deecd16f-6f89-49e6-9cc3-486cb2362482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:53.419926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:53.422097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:53.433693 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1306/2633] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:57:53.488276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1328/2634] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:53.495913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:53.497549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:53.582516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f24fe-d64a-407a-b404-e06cea7d5b54 service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:53.588040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:53.588891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.589157 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.589425 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.590049 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:53.591975 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:53.630028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1307/2635] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:57:53 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:53.634078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecd16f-6f89-49e6-9cc3-486cb2362482 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:53.646256 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecd16f-6f89-49e6-9cc3-486cb2362482 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:53.711960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:53.712624 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.712904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.713181 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.713817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:53.716155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:53.754446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1329/2636] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:57:53 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:57:53.759329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:53.777904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b04e2c-fe0d-4fbb-a7a5-2532df702f21 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:53.787185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:53.788233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.788785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.789299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.790734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:53.794068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:53.823893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1308/2637] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:53.825391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:53.837262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054d3ea-b887-4428-9705-5791479f6a50 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:53.842561 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:53.843435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.843748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.844001 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] PATH_INFO: `/v3/projects/52bab08340a44d41bd50d25a6b1514f1/users/0b69cf1ce40e4420ab7000a360d4d158/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.862803 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bab08340a44d41bd50d25a6b1514f1, user_id=0b69cf1ce40e4420ab7000a360d4d158, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:53.866058 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:53.882145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1330/2638] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:53 2022] PUT /identity/v3/projects/52bab08340a44d41bd50d25a6b1514f1/users/0b69cf1ce40e4420ab7000a360d4d158/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:57:53.886453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:53.897279 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2e340-c34e-46d8-ac8c-b9dfa1b51404 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:53.968560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f96338-f0af-4ab6-844e-d60930f71469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:53.969695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f96338-f0af-4ab6-844e-d60930f71469 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:53.969972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f96338-f0af-4ab6-844e-d60930f71469 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:53.972638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f96338-f0af-4ab6-844e-d60930f71469 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:53.973511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f96338-f0af-4ab6-844e-d60930f71469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:53.975493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f96338-f0af-4ab6-844e-d60930f71469 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:53.988317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1309/2639] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:57:53.991827 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:54.002391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9da96-fefd-4a75-b031-3d03d48e2c7a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:54.006869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:54.007522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:54.007793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:54.008060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] PATH_INFO: `/v3/projects/52bab08340a44d41bd50d25a6b1514f1/users/0b69cf1ce40e4420ab7000a360d4d158/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:54.010023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bab08340a44d41bd50d25a6b1514f1, user_id=0b69cf1ce40e4420ab7000a360d4d158, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:54.012778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:54.023413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1331/2640] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:57:53 2022] PUT /identity/v3/projects/52bab08340a44d41bd50d25a6b1514f1/users/0b69cf1ce40e4420ab7000a360d4d158/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:57:58.706390 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9807b-990a-4aa5-9a94-d5838b89d67e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:58.706900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9807b-990a-4aa5-9a94-d5838b89d67e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:58.706997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9807b-990a-4aa5-9a94-d5838b89d67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:58.752459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-8df9807b-990a-4aa5-9a94-d5838b89d67e None None] MFA Rules not processed for user `0b69cf1ce40e4420ab7000a360d4d158`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:57:58.753672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-8df9807b-990a-4aa5-9a94-d5838b89d67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:57:58.770609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:58.832464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1310/2641] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:57:58.845923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:57:58.889820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:57:58.923341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab095f4-ccec-41eb-afb9-3ce968ef0934 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:57:58.929487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:57:58.930135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:57:58.930559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:57:58.930729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:57:58.996234 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:57:58.997855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:57:59.008235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1332/2642] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:57:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:00.128701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f1501-eac5-46fa-9d82-bf536fa56417 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:00.129217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f1501-eac5-46fa-9d82-bf536fa56417 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:00.129305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f1501-eac5-46fa-9d82-bf536fa56417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:00.150600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d79f1501-eac5-46fa-9d82-bf536fa56417 None None] MFA Rules not processed for user `f16bdb81c5a249bb9f95e390dbd0dbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:00.151803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d79f1501-eac5-46fa-9d82-bf536fa56417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:00.159073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1311/2643] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:58:01.136423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:01.208722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d28d7e-6f3b-4cc2-a25b-6fe7c505caf7 service nova] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:01.213418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:01.214074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:01.214329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:01.214597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] PATH_INFO: `/v3/users/45f446d513e94ccba51391c7d6b03876` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:01.215711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f446d513e94ccba51391c7d6b03876)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:01.217704 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:01.278812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f446d513e94ccba51391c7d6b03876'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.279537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f446d513e94ccba51391c7d6b03876'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.280320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f446d513e94ccba51391c7d6b03876'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.288074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f446d513e94ccba51391c7d6b03876'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.297103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f446d513e94ccba51391c7d6b03876'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.313897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:01.322974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1333/2644] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:58:01 2022] DELETE /identity/v3/users/45f446d513e94ccba51391c7d6b03876 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:01.327509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:01.327981 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:01.328081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:01.363519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:01.364962 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:01.430326 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1312/2645] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:01.441806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:01.443501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:01.508859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6bb1e-eae6-459e-9b61-8e5f67c1e7f8 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:01.512531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b534a-5aa7-47b7-9332-d01ba6984b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:01.513143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b534a-5aa7-47b7-9332-d01ba6984b1b service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:01.513423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b534a-5aa7-47b7-9332-d01ba6984b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:01.513692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b534a-5aa7-47b7-9332-d01ba6984b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:01.523276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b534a-5aa7-47b7-9332-d01ba6984b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:01.525122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b534a-5aa7-47b7-9332-d01ba6984b1b service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:01.535922 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1334/2646] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:01.864570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:01.866372 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:01.886762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699b59-e33e-4e2b-b050-09ba9a720e33 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:01.893984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:01.895481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:01.896102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:01.896272 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] PATH_INFO: `/v3/projects/a952ae3bd5e14e39be82baa02aecf77d` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:01.897511 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a952ae3bd5e14e39be82baa02aecf77d)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:01.899816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:01.949776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a952ae3bd5e14e39be82baa02aecf77d', 'request_id': 'req-44b755ed-c74b-4ffd-bfb7-37383d048649'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.957128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a952ae3bd5e14e39be82baa02aecf77d', 'request_id': 'req-44b755ed-c74b-4ffd-bfb7-37383d048649'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.965776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a952ae3bd5e14e39be82baa02aecf77d', 'request_id': 'req-44b755ed-c74b-4ffd-bfb7-37383d048649'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.980501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] The token cache is being invalidate because project a952ae3bd5e14e39be82baa02aecf77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:58:01.980501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.981296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-44b755ed-c74b-4ffd-bfb7-37383d048649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:01.983087 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1313/2647] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:58:01 2022] DELETE /identity/v3/projects/a952ae3bd5e14e39be82baa02aecf77d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:01.988928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70021b-1062-4aef-9937-ef5ba0be35f7 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:01.989208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70021b-1062-4aef-9937-ef5ba0be35f7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:01.989482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70021b-1062-4aef-9937-ef5ba0be35f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.010739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2b70021b-1062-4aef-9937-ef5ba0be35f7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:02.012098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2b70021b-1062-4aef-9937-ef5ba0be35f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:02.083668 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1335/2648] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:58:02.085481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bec646-6578-4a36-8aa6-870cd00751f3 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.085759 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bec646-6578-4a36-8aa6-870cd00751f3 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.086028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bec646-6578-4a36-8aa6-870cd00751f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.106828 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-94bec646-6578-4a36-8aa6-870cd00751f3 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:02.108053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-94bec646-6578-4a36-8aa6-870cd00751f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:02.116159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1314/2649] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:02.120691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70021b-1062-4aef-9937-ef5ba0be35f7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.130543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70021b-1062-4aef-9937-ef5ba0be35f7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.135096 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.135703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.135977 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.136246 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.136933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:58:02.137489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.138877 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.163885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1336/2650] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:58:02.172455 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bec646-6578-4a36-8aa6-870cd00751f3 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.182415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bec646-6578-4a36-8aa6-870cd00751f3 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.188803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.188803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.188803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.188803 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.189416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.194487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.221953 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:02.229211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1315/2651] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:58:02 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:02.233920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.244460 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafcc01-7dd2-490d-ae5d-23499fdec896 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.305170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.306183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.306620 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.307082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.307717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.309690 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.335856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1337/2652] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:58:02 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:58:02.340423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.348565 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7efb5-9025-449c-8487-41948c95ccf8 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.353342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.354073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.354396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.354765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.355481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.357188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.369329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1316/2653] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:58:02.372405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.382856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad706c3-a956-41c3-b61f-690737188b4f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.387517 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58571-c373-45a5-b76f-1d5c4aab90ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.388116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58571-c373-45a5-b76f-1d5c4aab90ca admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.388406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58571-c373-45a5-b76f-1d5c4aab90ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.388672 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58571-c373-45a5-b76f-1d5c4aab90ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.389303 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd58571-c373-45a5-b76f-1d5c4aab90ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.390895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd58571-c373-45a5-b76f-1d5c4aab90ca admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.402717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1338/2654] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:02.406713 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.417169 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270cf5b-27da-4143-b83f-ab72444e99ee admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.422299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.423046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.423443 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.423695 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] PATH_INFO: `/v3/projects/8f9cbacc184649528a50e15120aa895b/users/efcc7012c42d4571a763f2125a1c2cc8/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.442790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9cbacc184649528a50e15120aa895b, user_id=efcc7012c42d4571a763f2125a1c2cc8, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.445523 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.460132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1317/2655] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:02 2022] PUT /identity/v3/projects/8f9cbacc184649528a50e15120aa895b/users/efcc7012c42d4571a763f2125a1c2cc8/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:58:02.472054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe862d-7ab4-493b-b95e-1828714a8533 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.472375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe862d-7ab4-493b-b95e-1828714a8533 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.472650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe862d-7ab4-493b-b95e-1828714a8533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.479821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:02.507592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-dbbe862d-7ab4-493b-b95e-1828714a8533 None None] MFA Rules not processed for user `efcc7012c42d4571a763f2125a1c2cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:02.509304 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-dbbe862d-7ab4-493b-b95e-1828714a8533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:02.584161 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1339/2656] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:02.589063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.656209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147bfa5-4c23-4b5d-b7ca-15f429bffaff admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.661574 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.662182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.662447 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.662708 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.663330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.665262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.702667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1318/2657] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:58:02 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:02.705463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe862d-7ab4-493b-b95e-1828714a8533 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.717485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe862d-7ab4-493b-b95e-1828714a8533 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.796239 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.796899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.797173 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.797442 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.798093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.800229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.823946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1340/2658] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:58:02 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:58:02.828377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.839413 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603223e-5a20-4c73-8384-1c030b3fad07 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.844177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.844848 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.845118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.845406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.846217 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.847821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.862368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1319/2659] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:02.863586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.874218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e0a04-2731-449e-abbd-d52c2ec88c14 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.879411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.880169 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.880502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.880771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] PATH_INFO: `/v3/projects/7576f7a92cbe4ee3a362239aa1ae2e3a/users/95a73eac44fd4bd69552990101bba584/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:02.899273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7576f7a92cbe4ee3a362239aa1ae2e3a, user_id=95a73eac44fd4bd69552990101bba584, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:02.901997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:02.917144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1341/2660] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:02 2022] PUT /identity/v3/projects/7576f7a92cbe4ee3a362239aa1ae2e3a/users/95a73eac44fd4bd69552990101bba584/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:02.921239 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:02.932252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4569326-9757-4e2c-8eb9-9fcab0c05379 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:02.997469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34044c9-89b0-46cb-a12f-f45a2f9d3221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:02.998723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34044c9-89b0-46cb-a12f-f45a2f9d3221 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:02.999024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34044c9-89b0-46cb-a12f-f45a2f9d3221 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:02.999288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34044c9-89b0-46cb-a12f-f45a2f9d3221 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:03.000085 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34044c9-89b0-46cb-a12f-f45a2f9d3221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:03.001636 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34044c9-89b0-46cb-a12f-f45a2f9d3221 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:03.015396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1320/2661] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:03.020144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:03.031735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b273f5-b555-497c-9443-3db9c32f4e3b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:03.036978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:03.037657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:03.038120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:03.038398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] PATH_INFO: `/v3/projects/7576f7a92cbe4ee3a362239aa1ae2e3a/users/95a73eac44fd4bd69552990101bba584/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:03.040430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7576f7a92cbe4ee3a362239aa1ae2e3a, user_id=95a73eac44fd4bd69552990101bba584, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:03.043148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:03.054691 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1342/2662] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:03 2022] PUT /identity/v3/projects/7576f7a92cbe4ee3a362239aa1ae2e3a/users/95a73eac44fd4bd69552990101bba584/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:03.065796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d773-f2cf-474e-a918-d9d820806ba6 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:03.066066 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d773-f2cf-474e-a918-d9d820806ba6 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:03.066342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d773-f2cf-474e-a918-d9d820806ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:03.097601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-48a4d773-f2cf-474e-a918-d9d820806ba6 None None] MFA Rules not processed for user `95a73eac44fd4bd69552990101bba584`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:03.098729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-48a4d773-f2cf-474e-a918-d9d820806ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:03.123148 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:03.178589 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1321/2663] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:03.190033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:03.191872 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:03.248511 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:03.269483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b26f39-e6d3-4a4d-84e6-e768cebc6f54 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:03.276006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:03.276987 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:03.277470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:03.278155 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:03.355171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:03.356775 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:03.366893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1343/2664] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:09.169775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d773-f2cf-474e-a918-d9d820806ba6 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:09.180488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d773-f2cf-474e-a918-d9d820806ba6 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:09.184235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7efaec-c752-447b-9493-78e45fb26312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:09.184859 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7efaec-c752-447b-9493-78e45fb26312 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:09.185137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7efaec-c752-447b-9493-78e45fb26312 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:09.185408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7efaec-c752-447b-9493-78e45fb26312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:09.195067 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7efaec-c752-447b-9493-78e45fb26312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:09.196550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7efaec-c752-447b-9493-78e45fb26312 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:09.206185 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1322/2665] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:13.362821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=28 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:19.201854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=24 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:34.426933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:34.565850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702e29a-b4f9-41f1-8869-aadcac15c5ec service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:34.571218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:34.571991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:34.572372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:34.572724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] PATH_INFO: `/v3/users/efcc7012c42d4571a763f2125a1c2cc8` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:34.573954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efcc7012c42d4571a763f2125a1c2cc8)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:34.575948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:34.622868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efcc7012c42d4571a763f2125a1c2cc8'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:34.623503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efcc7012c42d4571a763f2125a1c2cc8'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:34.624078 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efcc7012c42d4571a763f2125a1c2cc8'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:34.630798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efcc7012c42d4571a763f2125a1c2cc8'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:34.638663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efcc7012c42d4571a763f2125a1c2cc8'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:34.656347 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1344/2666] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:58:34 2022] DELETE /identity/v3/users/efcc7012c42d4571a763f2125a1c2cc8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:34.661932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:34.662698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:34.662698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:34.713100 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:34.714212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:34.776155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1323/2667] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:34.787416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:34.788964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:34.833018 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:34.867269 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f22b65-1e83-4d4d-a0a7-12d5fc20ffe5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:34.871209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:34.871823 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:34.872087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:34.872379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:34.881584 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:34.882999 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:34.893556 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1345/2668] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:35.169032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:35.170661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:35.184081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3e0af-e669-42e8-b2d1-e66e808bd405 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:35.189807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:35.190550 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:35.190825 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:35.191112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] PATH_INFO: `/v3/projects/8f9cbacc184649528a50e15120aa895b` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:35.192281 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9cbacc184649528a50e15120aa895b)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:35.194221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:35.244501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9cbacc184649528a50e15120aa895b', 'request_id': 'req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.250008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9cbacc184649528a50e15120aa895b', 'request_id': 'req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.256807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9cbacc184649528a50e15120aa895b', 'request_id': 'req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.270507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] The token cache is being invalidate because project 8f9cbacc184649528a50e15120aa895b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:58:35.270788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.271383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.272902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1324/2669] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:58:35 2022] DELETE /identity/v3/projects/8f9cbacc184649528a50e15120aa895b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:35.277615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:35.287960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7d565-37bd-4006-a1b1-47d27fde8c94 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:35.346503 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:35.347176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:35.347446 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:35.347711 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] PATH_INFO: `/v3/users/95a73eac44fd4bd69552990101bba584` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:35.348856 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95a73eac44fd4bd69552990101bba584)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:35.350807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:35.400252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95a73eac44fd4bd69552990101bba584'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.400944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95a73eac44fd4bd69552990101bba584'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.401550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95a73eac44fd4bd69552990101bba584'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.409031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95a73eac44fd4bd69552990101bba584'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.418942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-79e572bb-382a-434a-bafa-1549ddfd1096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95a73eac44fd4bd69552990101bba584'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.439471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1346/2670] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:58:35 2022] DELETE /identity/v3/users/95a73eac44fd4bd69552990101bba584 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:35.841627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:35.841627 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:35.847869 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b4b27-ede4-42f1-9c8d-f40dd2fbb1d9 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:35.900841 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:35.907243 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:35.907883 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:35.908380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:35.908804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] PATH_INFO: `/v3/projects/7576f7a92cbe4ee3a362239aa1ae2e3a` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:35.910407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7576f7a92cbe4ee3a362239aa1ae2e3a)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:35.912384 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:35.959518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7576f7a92cbe4ee3a362239aa1ae2e3a', 'request_id': 'req-0607094d-09d8-43d6-bf36-84eb9f0e2aca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.964850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7576f7a92cbe4ee3a362239aa1ae2e3a', 'request_id': 'req-0607094d-09d8-43d6-bf36-84eb9f0e2aca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.971145 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7576f7a92cbe4ee3a362239aa1ae2e3a', 'request_id': 'req-0607094d-09d8-43d6-bf36-84eb9f0e2aca'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.986619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] The token cache is being invalidate because project 7576f7a92cbe4ee3a362239aa1ae2e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:58:35.987074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.987766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0607094d-09d8-43d6-bf36-84eb9f0e2aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:35.989383 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1325/2671] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:58:35 2022] DELETE /identity/v3/projects/7576f7a92cbe4ee3a362239aa1ae2e3a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:35.994533 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44a961-f827-4bc1-9581-6b79df703916 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:35.994866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44a961-f827-4bc1-9581-6b79df703916 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:35.995139 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44a961-f827-4bc1-9581-6b79df703916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.046137 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0b44a961-f827-4bc1-9581-6b79df703916 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:36.047623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0b44a961-f827-4bc1-9581-6b79df703916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:36.061656 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:36.109751 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1347/2672] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:36.117798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03db2-d435-4168-b744-0902a6a12f1b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.118080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03db2-d435-4168-b744-0902a6a12f1b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.118355 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03db2-d435-4168-b744-0902a6a12f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.138684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d2d03db2-d435-4168-b744-0902a6a12f1b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:36.139918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d2d03db2-d435-4168-b744-0902a6a12f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:36.148353 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1326/2673] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:36.153067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44a961-f827-4bc1-9581-6b79df703916 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.162995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44a961-f827-4bc1-9581-6b79df703916 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.168291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.168913 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.169185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.169448 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.170093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:58:36.170385 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.172084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.184059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1348/2674] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:58:36.188530 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d03db2-d435-4168-b744-0902a6a12f1b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.198037 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d03db2-d435-4168-b744-0902a6a12f1b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.203485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.204080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.204388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.204659 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.205291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.207199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.237763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1327/2675] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:58:36 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:36.242142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.252243 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad538-7949-4834-b5f9-6a6009fb12a2 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.310306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.310921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.311202 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.311467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.312084 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.314119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.340504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1349/2676] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:58:36 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:36.344800 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.357033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620554df-4a0c-45ef-a80e-d7d676b09c5b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.372127 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.372778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.373151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.373401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.374235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.375925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.386651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1328/2677] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:36.390992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.401798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0256091-72fd-4feb-a92d-47a7b80e8a20 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.407554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5b607-5658-4e5e-9733-88726d08d07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.408164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b607-5658-4e5e-9733-88726d08d07d admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.408481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b607-5658-4e5e-9733-88726d08d07d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.408745 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b607-5658-4e5e-9733-88726d08d07d admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.409391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5b607-5658-4e5e-9733-88726d08d07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.411076 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5b607-5658-4e5e-9733-88726d08d07d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.423177 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1350/2678] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:36.427682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.437584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed13736-6b90-4ab1-a662-cbc58a40dbf3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.442323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.442983 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.443254 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.443518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] PATH_INFO: `/v3/projects/1debbd4111794112aa4101aadac5a5a6/users/c4f80557d1b440a79fcc0d8215a0a26e/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.459692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1debbd4111794112aa4101aadac5a5a6, user_id=c4f80557d1b440a79fcc0d8215a0a26e, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.462507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.477811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1329/2679] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:36 2022] PUT /identity/v3/projects/1debbd4111794112aa4101aadac5a5a6/users/c4f80557d1b440a79fcc0d8215a0a26e/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:36.488463 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78cd70-2614-42c2-a747-9280fe5391cc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.488974 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78cd70-2614-42c2-a747-9280fe5391cc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.489086 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78cd70-2614-42c2-a747-9280fe5391cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.521195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0e78cd70-2614-42c2-a747-9280fe5391cc None None] MFA Rules not processed for user `c4f80557d1b440a79fcc0d8215a0a26e`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:36.523020 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0e78cd70-2614-42c2-a747-9280fe5391cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:36.593713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1351/2680] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:36.598683 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.663598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badca7f3-3b0d-4cab-861f-c755e24cf58e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.669381 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.670006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.670288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.670551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.671181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.673181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.704652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1330/2681] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:58:36 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:36.709160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78cd70-2614-42c2-a747-9280fe5391cc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.721739 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78cd70-2614-42c2-a747-9280fe5391cc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.733776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:36.783582 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.784208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.784555 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.784824 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.785480 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.787432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.811254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1352/2682] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:58:36 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 10:58:36.815970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.826874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f3ddf-9af5-4edd-8b71-b04b96c8638c admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.832249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.832893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.833165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.833469 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.834119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.835732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.838119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:36.851638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1331/2683] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:58:36.852961 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.864106 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8b278-483d-46f8-bc5d-12afced7e988 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.869192 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.869941 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.870278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.870639 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] PATH_INFO: `/v3/projects/834446d2652b4b42add6195f2be7148a/users/fefb2bdcf72d42e59b37c41a6cc3d89a/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.886393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834446d2652b4b42add6195f2be7148a, user_id=fefb2bdcf72d42e59b37c41a6cc3d89a, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.889067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.903661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1353/2684] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:36 2022] PUT /identity/v3/projects/834446d2652b4b42add6195f2be7148a/users/fefb2bdcf72d42e59b37c41a6cc3d89a/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:58:36.908165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:36.918091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722b94d-bd87-4341-a5f9-a43b71f7b03b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:36.979865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e7c98-4a0c-49e4-b234-2c2e4f493f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:36.980833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e7c98-4a0c-49e4-b234-2c2e4f493f41 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:36.981109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e7c98-4a0c-49e4-b234-2c2e4f493f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:36.981377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e7c98-4a0c-49e4-b234-2c2e4f493f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:36.982481 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e7c98-4a0c-49e4-b234-2c2e4f493f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:36.983623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e7c98-4a0c-49e4-b234-2c2e4f493f41 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:36.994466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1332/2685] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:36.998976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:37.010191 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219cff0-7df6-4f36-81ac-f1901f1b2848 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:37.018743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:37.019482 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:37.019814 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:37.020148 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] PATH_INFO: `/v3/projects/834446d2652b4b42add6195f2be7148a/users/fefb2bdcf72d42e59b37c41a6cc3d89a/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:37.022589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834446d2652b4b42add6195f2be7148a, user_id=fefb2bdcf72d42e59b37c41a6cc3d89a, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:37.025679 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:37.042870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1354/2686] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:36 2022] PUT /identity/v3/projects/834446d2652b4b42add6195f2be7148a/users/fefb2bdcf72d42e59b37c41a6cc3d89a/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:37.056791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e284a-3d2c-4e4b-897f-f396ae26b0ed None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:37.057165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e284a-3d2c-4e4b-897f-f396ae26b0ed None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:37.057584 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e284a-3d2c-4e4b-897f-f396ae26b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:37.091682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d06e284a-3d2c-4e4b-897f-f396ae26b0ed None None] MFA Rules not processed for user `fefb2bdcf72d42e59b37c41a6cc3d89a`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:37.092899 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d06e284a-3d2c-4e4b-897f-f396ae26b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:37.171616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1333/2687] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 10:58:37.188034 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:37.190509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:37.256416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296c06-f464-48de-a009-f36be43c6a71 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:37.260546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:37.261141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:37.261420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:37.261687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:37.274768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:37.276945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:37.288244 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1355/2688] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:37.377490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e284a-3d2c-4e4b-897f-f396ae26b0ed None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:37.388568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e284a-3d2c-4e4b-897f-f396ae26b0ed None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:37.392855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:37.393451 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:37.393728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:37.393996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:37.469482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:37.470851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:37.480208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1334/2689] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:39.601944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:39.603660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:39.689129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c11b82-c5ed-4977-ae15-498678c79106 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:39.693603 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b22ea-ce5e-49b6-9e47-1d80fb76882f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:39.694204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b22ea-ce5e-49b6-9e47-1d80fb76882f service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:39.694474 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b22ea-ce5e-49b6-9e47-1d80fb76882f service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:39.694812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b22ea-ce5e-49b6-9e47-1d80fb76882f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:39.697081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-597b22ea-ce5e-49b6-9e47-1d80fb76882f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:39.702057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:39.770959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b22ea-ce5e-49b6-9e47-1d80fb76882f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:39.772428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b22ea-ce5e-49b6-9e47-1d80fb76882f service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:39.782927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1356/2690] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:42.321011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:42.343745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:42.388672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082542-a0c0-464a-9847-3e9b6c8f99f0 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:42.393462 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:42.394143 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:42.394352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:42.394612 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] PATH_INFO: `/v3/users/c4f80557d1b440a79fcc0d8215a0a26e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:42.395723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f80557d1b440a79fcc0d8215a0a26e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:42.397680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:42.447284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f80557d1b440a79fcc0d8215a0a26e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:42.456743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f80557d1b440a79fcc0d8215a0a26e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:42.457458 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f80557d1b440a79fcc0d8215a0a26e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:42.465928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f80557d1b440a79fcc0d8215a0a26e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:42.478782 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f80557d1b440a79fcc0d8215a0a26e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:42.485050 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1335/2691] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:58:42 2022] DELETE /identity/v3/users/c4f80557d1b440a79fcc0d8215a0a26e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:42.489749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:42.490259 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:42.490378 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:42.511998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:42.513227 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:42.573490 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1357/2692] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:42.584598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:42.586162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:42.652561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba18de-12fb-4e6d-9d04-f03d46c49158 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:42.656719 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:42.657342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:42.657613 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:42.657878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:42.667101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:42.668591 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:42.678936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1336/2693] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:42.956405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:42.957929 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:42.970325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7f29b-3091-4a86-8f2a-0961f8f6eefc None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:42.976437 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:42.977071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:42.977338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:42.977598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] PATH_INFO: `/v3/projects/1debbd4111794112aa4101aadac5a5a6` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:42.978805 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1debbd4111794112aa4101aadac5a5a6)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:42.980752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:43.034408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1debbd4111794112aa4101aadac5a5a6', 'request_id': 'req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.041014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1debbd4111794112aa4101aadac5a5a6', 'request_id': 'req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.047402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1debbd4111794112aa4101aadac5a5a6', 'request_id': 'req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.061595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] The token cache is being invalidate because project 1debbd4111794112aa4101aadac5a5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:58:43.061898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.062678 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.064332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1358/2694] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:58:42 2022] DELETE /identity/v3/projects/1debbd4111794112aa4101aadac5a5a6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:43.068485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:43.078541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490211-27fe-4dcf-9880-acb0b4964080 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:43.116773 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:43.136189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:43.136852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:43.137125 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:43.137394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] PATH_INFO: `/v3/users/fefb2bdcf72d42e59b37c41a6cc3d89a` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:43.138520 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fefb2bdcf72d42e59b37c41a6cc3d89a)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:43.140537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:43.191786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fefb2bdcf72d42e59b37c41a6cc3d89a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.198489 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fefb2bdcf72d42e59b37c41a6cc3d89a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.199211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fefb2bdcf72d42e59b37c41a6cc3d89a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.208200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fefb2bdcf72d42e59b37c41a6cc3d89a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.221082 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7d18f875-641c-40db-b1a8-d91d9baca296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fefb2bdcf72d42e59b37c41a6cc3d89a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.227074 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1337/2695] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:58:43 2022] DELETE /identity/v3/users/fefb2bdcf72d42e59b37c41a6cc3d89a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:43.511184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:43.512795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:43.527091 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c21bb-2896-4232-8c7e-b324cd32fcfd admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:43.550267 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:43.589991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:43.590558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:43.590834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:43.591101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] PATH_INFO: `/v3/projects/834446d2652b4b42add6195f2be7148a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:43.592398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834446d2652b4b42add6195f2be7148a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:43.594451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:43.644989 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834446d2652b4b42add6195f2be7148a', 'request_id': 'req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.651645 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834446d2652b4b42add6195f2be7148a', 'request_id': 'req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.658043 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834446d2652b4b42add6195f2be7148a', 'request_id': 'req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.673033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] The token cache is being invalidate because project 834446d2652b4b42add6195f2be7148a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:58:43.673318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.673948 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-621d1ab4-6d1c-407b-b9ca-900b5bcd8156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:58:43.675453 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1359/2696] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:58:43 2022] DELETE /identity/v3/projects/834446d2652b4b42add6195f2be7148a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:43.680517 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e246-302f-4220-bf5b-85dda161cb66 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:43.680998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e246-302f-4220-bf5b-85dda161cb66 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:43.681099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e246-302f-4220-bf5b-85dda161cb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:43.700912 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a9a9e246-302f-4220-bf5b-85dda161cb66 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:43.702677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a9a9e246-302f-4220-bf5b-85dda161cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:43.765111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1338/2697] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:43.773911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c643f-95b4-4f00-93b3-1462fa661e3f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:43.774182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c643f-95b4-4f00-93b3-1462fa661e3f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:43.774440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c643f-95b4-4f00-93b3-1462fa661e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:43.798056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-233c643f-95b4-4f00-93b3-1462fa661e3f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:43.799283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-233c643f-95b4-4f00-93b3-1462fa661e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:43.808195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1360/2698] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:43.812780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9e246-302f-4220-bf5b-85dda161cb66 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:43.822985 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9e246-302f-4220-bf5b-85dda161cb66 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:43.827820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:43.828554 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:43.828995 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:43.829262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:43.829902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:58:43.830194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:43.831798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:43.843515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1339/2699] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:58:43.848135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c643f-95b4-4f00-93b3-1462fa661e3f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:43.857807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c643f-95b4-4f00-93b3-1462fa661e3f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:43.864307 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:43.864975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:43.865248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:43.865508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:43.866130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:43.868041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:43.898702 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1361/2700] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:58:43 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:43.903466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:43.914327 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea4d87-2f33-479e-9a28-e5b298281564 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:43.961155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:43.977095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:43.977716 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:43.977987 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:43.978250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:43.978875 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:43.980868 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:44.004649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1340/2701] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:58:43 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:44.009041 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:44.020540 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a14ab2-113e-4de8-a234-2f71cbac234a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:44.026028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:44.026661 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:44.026935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:44.027226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:44.027865 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:44.029504 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:44.039717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1362/2702] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:44.043965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:44.054984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911e154-04ee-4635-a2e2-ae5e2833a82a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:44.060675 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404d9c1-3f96-4b4b-8ba8-48573242d45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:44.061262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404d9c1-3f96-4b4b-8ba8-48573242d45a admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:44.061538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404d9c1-3f96-4b4b-8ba8-48573242d45a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:44.061823 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404d9c1-3f96-4b4b-8ba8-48573242d45a admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:44.062700 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404d9c1-3f96-4b4b-8ba8-48573242d45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:44.064510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404d9c1-3f96-4b4b-8ba8-48573242d45a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:44.076622 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1341/2703] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:44.081685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:44.096695 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8e1ed-1b53-452f-ac57-67e4bb2f5f7c admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:44.102568 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:44.103215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:44.103481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:44.103748 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] PATH_INFO: `/v3/projects/0b225833077e4ab2b962af3f53dbf6e2/users/92ccb81f199648af9a95b9723e083659/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:44.120183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b225833077e4ab2b962af3f53dbf6e2, user_id=92ccb81f199648af9a95b9723e083659, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:44.122875 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:44.139411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1363/2704] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:44 2022] PUT /identity/v3/projects/0b225833077e4ab2b962af3f53dbf6e2/users/92ccb81f199648af9a95b9723e083659/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:44.145112 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13403a-26ea-4ff6-9381-ac78ed8baba7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:44.145382 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13403a-26ea-4ff6-9381-ac78ed8baba7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:44.145673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13403a-26ea-4ff6-9381-ac78ed8baba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:44.165577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-aa13403a-26ea-4ff6-9381-ac78ed8baba7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:44.166784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-aa13403a-26ea-4ff6-9381-ac78ed8baba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:44.228033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1342/2705] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:44.238612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:44.240654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:44.309358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa250133-1552-44d2-bf7c-f261fd34bfa5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:44.314116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:44.314718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:44.314991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:44.315255 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:44.321352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:44.325254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:44.327100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:44.337925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1364/2706] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:44.998285 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f7f43-3751-47e3-a427-4f2217c5612e None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:44.998871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f7f43-3751-47e3-a427-4f2217c5612e None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:44.999047 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f7f43-3751-47e3-a427-4f2217c5612e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:45.029079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6f5f7f43-3751-47e3-a427-4f2217c5612e None None] MFA Rules not processed for user `92ccb81f199648af9a95b9723e083659`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:45.030339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6f5f7f43-3751-47e3-a427-4f2217c5612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:45.095457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1343/2707] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:45.100047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:45.110252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fa0c6-241f-47c0-a345-a80849b5d041 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:45.115275 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:45.115907 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:45.116135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:45.116434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:45.117059 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:45.119089 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:45.149458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1365/2708] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:58:45 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:45.154300 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f7f43-3751-47e3-a427-4f2217c5612e None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:45.166724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f7f43-3751-47e3-a427-4f2217c5612e None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:45.221093 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:45.231123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:45.231799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:45.232731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:45.233505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:45.233945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:45.236810 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:45.263010 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1344/2709] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:58:45 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:58:45.267622 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:45.286119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eb10b-6485-4c8a-b77a-0899d2148d2d admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:45.291223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:45.291832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:45.292102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:45.292405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:45.293048 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:45.294675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:45.306215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1366/2710] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:45.310648 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:45.321753 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c1d75-35d0-48d7-aab7-8fce8553c279 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:45.329079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:45.329752 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:45.330032 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:45.330307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] PATH_INFO: `/v3/projects/f60cb6b3498f46668dbfda5f0f0e64c8/users/d316e73fee16445d83e7c4393b5035c3/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:45.350886 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60cb6b3498f46668dbfda5f0f0e64c8, user_id=d316e73fee16445d83e7c4393b5035c3, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:45.353672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:45.370748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1345/2711] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:45 2022] PUT /identity/v3/projects/f60cb6b3498f46668dbfda5f0f0e64c8/users/d316e73fee16445d83e7c4393b5035c3/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:45.375734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:45.386224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337c23a-bf1f-4df8-97ae-9fff0a4fc1a3 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:45.447735 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b7041-8e9a-4d65-8064-102cca5b1dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:45.448294 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b7041-8e9a-4d65-8064-102cca5b1dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:45.448562 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b7041-8e9a-4d65-8064-102cca5b1dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:45.449655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b7041-8e9a-4d65-8064-102cca5b1dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:45.450342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b7041-8e9a-4d65-8064-102cca5b1dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:45.451967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b7041-8e9a-4d65-8064-102cca5b1dd1 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:45.462578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1367/2712] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:58:45.466916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:45.477914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527799d-efb9-4f11-a221-29a6c9faccca admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:45.483011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:45.483658 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:45.483933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:45.484205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] PATH_INFO: `/v3/projects/f60cb6b3498f46668dbfda5f0f0e64c8/users/d316e73fee16445d83e7c4393b5035c3/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:45.486330 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60cb6b3498f46668dbfda5f0f0e64c8, user_id=d316e73fee16445d83e7c4393b5035c3, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:45.489020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:45.499890 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1346/2713] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:58:45 2022] PUT /identity/v3/projects/f60cb6b3498f46668dbfda5f0f0e64c8/users/d316e73fee16445d83e7c4393b5035c3/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:58:46.158914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:46.159452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:46.159452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:46.191455 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] MFA Rules not processed for user `d316e73fee16445d83e7c4393b5035c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:58:46.192722 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:46.272762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1368/2714] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:58:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:58:46.285539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:46.287506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:46.377377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ecce-51ef-4b0e-9a05-d709bcdb1a7b admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:46.381505 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:46.382131 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:46.382404 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:46.382672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:46.442109 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:46.443569 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:46.452221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1347/2715] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:47.830425 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:58:47.832054 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:58:47.845440 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba995b6-0fc6-4939-941c-df9f4ee72145 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:58:47.849979 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:58:47.850577 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:58:47.850849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:58:47.851116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:58:47.860786 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:58:47.862215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:58:47.872022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1369/2716] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:58:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:58:56.448485 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:58:57.867795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:01.478579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:01.549741 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83082ec-66e3-4900-9262-ec6f87b1681e service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:01.554733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:01.555411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:01.555733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:01.556001 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] PATH_INFO: `/v3/users/92ccb81f199648af9a95b9723e083659` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:01.557180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92ccb81f199648af9a95b9723e083659)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:01.559099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:01.619712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92ccb81f199648af9a95b9723e083659'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.627023 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92ccb81f199648af9a95b9723e083659'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.627728 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92ccb81f199648af9a95b9723e083659'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.639159 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92ccb81f199648af9a95b9723e083659'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.655437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92ccb81f199648af9a95b9723e083659'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.661561 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1348/2717] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:59:01 2022] DELETE /identity/v3/users/92ccb81f199648af9a95b9723e083659 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:59:01.842817 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:01.858576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfea8-1fe8-4f81-ae23-05b77542caca service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:01.918251 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:01.918796 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:01.919065 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:01.919330 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] PATH_INFO: `/v3/projects/0b225833077e4ab2b962af3f53dbf6e2` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:01.920508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b225833077e4ab2b962af3f53dbf6e2)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:01.922413 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:01.970475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b225833077e4ab2b962af3f53dbf6e2', 'request_id': 'req-c88f33a5-ba57-4ea4-b815-5eac7243be15'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.976467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b225833077e4ab2b962af3f53dbf6e2', 'request_id': 'req-c88f33a5-ba57-4ea4-b815-5eac7243be15'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.982476 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b225833077e4ab2b962af3f53dbf6e2', 'request_id': 'req-c88f33a5-ba57-4ea4-b815-5eac7243be15'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.997029 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] The token cache is being invalidate because project 0b225833077e4ab2b962af3f53dbf6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:59:01.997297 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.998006 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:01.999658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1370/2718] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:59:01 2022] DELETE /identity/v3/projects/0b225833077e4ab2b962af3f53dbf6e2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:59:02.004515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:02.006966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:02.023208 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc525a5-8e4b-4b77-8b70-09aec8beb229 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:02.088033 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:02.088702 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:02.088965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:02.089236 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] PATH_INFO: `/v3/users/d316e73fee16445d83e7c4393b5035c3` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:02.090409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d316e73fee16445d83e7c4393b5035c3)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:02.092424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:02.252492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:02.284840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd316e73fee16445d83e7c4393b5035c3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.292994 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd316e73fee16445d83e7c4393b5035c3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.294289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd316e73fee16445d83e7c4393b5035c3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.305054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd316e73fee16445d83e7c4393b5035c3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.317786 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-adcc052c-3349-42e7-a5af-f3e299f53654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd316e73fee16445d83e7c4393b5035c3'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.324338 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1349/2719] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:59:02 2022] DELETE /identity/v3/users/d316e73fee16445d83e7c4393b5035c3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 10:59:02.519791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:02.521450 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:02.540572 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f33a5-ba57-4ea4-b815-5eac7243be15 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:02.599144 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:02.599778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:02.600195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:02.600590 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] PATH_INFO: `/v3/projects/f60cb6b3498f46668dbfda5f0f0e64c8` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:02.601835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f60cb6b3498f46668dbfda5f0f0e64c8)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:02.603871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:02.644616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:02.663496 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f60cb6b3498f46668dbfda5f0f0e64c8', 'request_id': 'req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.670038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f60cb6b3498f46668dbfda5f0f0e64c8', 'request_id': 'req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.675623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f60cb6b3498f46668dbfda5f0f0e64c8', 'request_id': 'req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.689887 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] The token cache is being invalidate because project f60cb6b3498f46668dbfda5f0f0e64c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:59:02.690170 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.690776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-23bc06c3-f48a-4925-be8e-d8886b0dd6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:02.692383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1371/2720] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:59:02 2022] DELETE /identity/v3/projects/f60cb6b3498f46668dbfda5f0f0e64c8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:59:02.697501 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8fb6b-91b5-44cb-a2ef-1daa64fe0c70 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:02.698022 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8fb6b-91b5-44cb-a2ef-1daa64fe0c70 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:02.698721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8fb6b-91b5-44cb-a2ef-1daa64fe0c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:02.719917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-a8b8fb6b-91b5-44cb-a2ef-1daa64fe0c70 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:59:02.721147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a8b8fb6b-91b5-44cb-a2ef-1daa64fe0c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:02.780651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1350/2721] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:59:02.789364 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6993bb-954f-47fd-83c7-d929a535f6b7 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:02.789728 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6993bb-954f-47fd-83c7-d929a535f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:02.790062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6993bb-954f-47fd-83c7-d929a535f6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:02.816749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-3a6993bb-954f-47fd-83c7-d929a535f6b7 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:59:02.817958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3a6993bb-954f-47fd-83c7-d929a535f6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:02.827359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1372/2722] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:59:02.831711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8fb6b-91b5-44cb-a2ef-1daa64fe0c70 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:02.841673 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8fb6b-91b5-44cb-a2ef-1daa64fe0c70 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:02.846461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:02.847044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:02.847307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:02.847566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:02.848198 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 10:59:02.848528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:02.850135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:02.861711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1351/2723] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 10:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 10:59:02.865991 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6993bb-954f-47fd-83c7-d929a535f6b7 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:02.876056 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6993bb-954f-47fd-83c7-d929a535f6b7 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:02.881976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:02.882593 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:02.882866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:02.883127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:02.883778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:02.886172 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:02.922608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1373/2724] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 10:59:02 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:59:02.926835 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:02.937577 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770aebf-c9f7-475d-a676-5c2369ba04bd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:03.001979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:03.002585 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:03.002864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:03.003134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:03.003744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:03.006306 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:03.029555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1352/2725] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 10:59:02 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 10:59:03.033799 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:03.044299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6785-2472-4865-b053-cddddb481c1b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:03.049219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:03.049808 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:03.050079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:03.050345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:03.050980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:03.052595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:03.062479 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1374/2726] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 10:59:03.066635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:03.077537 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206c1f-0c58-4004-9f90-eef8acd88f79 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:03.083343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db24cc-304c-4ae5-a72b-7239c3ef8c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:03.083904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db24cc-304c-4ae5-a72b-7239c3ef8c99 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:03.084177 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db24cc-304c-4ae5-a72b-7239c3ef8c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:03.084465 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db24cc-304c-4ae5-a72b-7239c3ef8c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:03.085097 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db24cc-304c-4ae5-a72b-7239c3ef8c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:03.086694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db24cc-304c-4ae5-a72b-7239c3ef8c99 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:03.097411 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1353/2727] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 10:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 10:59:03.101629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:03.112203 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20066-26e8-48c1-8173-2ae9a81fae39 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:03.117407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:03.118055 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:03.118298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:03.118569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] PATH_INFO: `/v3/projects/4fe6498fa3124339a53ece5c5baa2cff/users/b86c0494dc7b4c0a8610f8de95f2ff83/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:03.134653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe6498fa3124339a53ece5c5baa2cff, user_id=b86c0494dc7b4c0a8610f8de95f2ff83, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:03.137904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:03.156075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1375/2728] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 10:59:03 2022] PUT /identity/v3/projects/4fe6498fa3124339a53ece5c5baa2cff/users/b86c0494dc7b4c0a8610f8de95f2ff83/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:59:03.171546 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376184-4781-4906-ba91-991f53ec0de7 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:03.171826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376184-4781-4906-ba91-991f53ec0de7 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:03.172119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376184-4781-4906-ba91-991f53ec0de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:03.182164 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:03.203053 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-2d376184-4781-4906-ba91-991f53ec0de7 None None] MFA Rules not processed for user `b86c0494dc7b4c0a8610f8de95f2ff83`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:59:03.204247 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2d376184-4781-4906-ba91-991f53ec0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:03.267316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1354/2729] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:59:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:59:03.280023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:03.283258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:03.346849 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c5359-3f74-4545-a282-86654cbb22df admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:03.350860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482d7f2-5b89-436b-8633-3e9fc6ee783d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:03.351483 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482d7f2-5b89-436b-8633-3e9fc6ee783d service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:03.351763 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482d7f2-5b89-436b-8633-3e9fc6ee783d service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:03.352032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482d7f2-5b89-436b-8633-3e9fc6ee783d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:03.361168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482d7f2-5b89-436b-8633-3e9fc6ee783d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:03.362599 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482d7f2-5b89-436b-8633-3e9fc6ee783d service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:03.371515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1376/2730] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:59:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:59:06.504788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d376184-4781-4906-ba91-991f53ec0de7 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:06.571290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d376184-4781-4906-ba91-991f53ec0de7 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:06.576980 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:06.577599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:06.577891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:06.578151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] PATH_INFO: `/v3/users/b86c0494dc7b4c0a8610f8de95f2ff83` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:06.579385 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86c0494dc7b4c0a8610f8de95f2ff83)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:06.581386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:06.631474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86c0494dc7b4c0a8610f8de95f2ff83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:06.638261 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86c0494dc7b4c0a8610f8de95f2ff83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:06.638932 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86c0494dc7b4c0a8610f8de95f2ff83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:06.648155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86c0494dc7b4c0a8610f8de95f2ff83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:06.660670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86c0494dc7b4c0a8610f8de95f2ff83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:06.666396 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1355/2731] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 10:59:06 2022] DELETE /identity/v3/users/b86c0494dc7b4c0a8610f8de95f2ff83 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:59:06.671216 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:06.671733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:06.671733 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:06.694164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:59:06.695428 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:06.757452 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:06.768000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1377/2732] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:59:06.778619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:06.780525 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:06.796760 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:06.843999 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490eb31-5d49-4159-9330-91ad70691ba0 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:06.848376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:06.849190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:06.849467 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:06.849734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:06.859227 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:06.860691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:06.870325 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1356/2733] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:59:07.155421 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:07.156868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:07.170848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b798f-a0d9-4f3d-9cfd-67db7854ac04 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:07.177280 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:07.177976 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:07.178510 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:07.178843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] PATH_INFO: `/v3/projects/4fe6498fa3124339a53ece5c5baa2cff` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:07.180398 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe6498fa3124339a53ece5c5baa2cff)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:07.182673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:07.230911 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe6498fa3124339a53ece5c5baa2cff', 'request_id': 'req-fb901354-2f38-4c9c-863e-ac79ad0154fd'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:07.237288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe6498fa3124339a53ece5c5baa2cff', 'request_id': 'req-fb901354-2f38-4c9c-863e-ac79ad0154fd'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:07.242890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe6498fa3124339a53ece5c5baa2cff', 'request_id': 'req-fb901354-2f38-4c9c-863e-ac79ad0154fd'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:07.257231 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] The token cache is being invalidate because project 4fe6498fa3124339a53ece5c5baa2cff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 10:59:07.257520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:07.258135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-fb901354-2f38-4c9c-863e-ac79ad0154fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 10:59:07.259631 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1378/2734] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 10:59:07 2022] DELETE /identity/v3/projects/4fe6498fa3124339a53ece5c5baa2cff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 10:59:14.997685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:14.999297 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:15.096628 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0397bc-4caf-4448-abef-532245077ff6 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:15.161960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:15.162570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:15.162827 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:15.163081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:15.164880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:15.176614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:15.178400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:15.188969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1357/2735] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:59:17.254115 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:18.796394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d2e7-267a-45c9-8f1d-049ff76faab0 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:18.797003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d2e7-267a-45c9-8f1d-049ff76faab0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:18.797003 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d2e7-267a-45c9-8f1d-049ff76faab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:18.817621 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-4e62d2e7-267a-45c9-8f1d-049ff76faab0 None None] MFA Rules not processed for user `5cde581444394ae0b342d0c1ec435c76`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:59:18.818860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4e62d2e7-267a-45c9-8f1d-049ff76faab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:18.881986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1379/2736] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:59:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:59:25.184453 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=87 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:26.412846 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 10:59:26.415459 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:26.478989 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93bf72-e781-4431-80ae-24627797a79c service placement] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 10:59:26.483046 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 10:59:26.483649 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:26.483917 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:26.484181 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:26.486069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:26.546714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 10:59:26.548200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 10:59:26.559232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1358/2737] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 10:59:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 10:59:28.870102 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=25 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:33.496558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f06e-7578-4f1f-b14e-47908766a4ce None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 10:59:33.497256 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f06e-7578-4f1f-b14e-47908766a4ce None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 10:59:33.497256 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27f06e-7578-4f1f-b14e-47908766a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 10:59:33.519260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5d27f06e-7578-4f1f-b14e-47908766a4ce None None] MFA Rules not processed for user `922f49fa7d324ebdade043e1be1a476f`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 10:59:33.520793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5d27f06e-7578-4f1f-b14e-47908766a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 10:59:33.528921 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1380/2738] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 10:59:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 10:59:36.553118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=48 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 10:59:43.513889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:00:19.241604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:19.253151 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd3a36-dd79-4289-8001-05b1e1d2682c service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:19.258878 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:19.259926 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:19.260111 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:19.260543 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:19.263476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:19.367315 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:19.368862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:19.388927 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1359/2739] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:00:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:00:27.582552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f06e-7578-4f1f-b14e-47908766a4ce None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:27.593898 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27f06e-7578-4f1f-b14e-47908766a4ce None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:27.598724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:27.599319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:27.599589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:27.599853 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:27.602248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:27.617869 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:27.619698 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:27.630831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1381/2740] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:00:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:00:29.386265 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=46 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:00:37.625789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=6 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:00:45.056081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:45.058606 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:45.132655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b878c-e7ab-48d0-b18c-e0eb882ba006 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:45.137534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:45.138229 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.138506 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.139559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] PATH_INFO: `/v3/users/5cde581444394ae0b342d0c1ec435c76` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.145079 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cde581444394ae0b342d0c1ec435c76)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:45.147450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:45.206248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cde581444394ae0b342d0c1ec435c76'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.212880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cde581444394ae0b342d0c1ec435c76'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.213524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cde581444394ae0b342d0c1ec435c76'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.221547 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cde581444394ae0b342d0c1ec435c76'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.232928 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-463022ff-c436-439c-8392-296e016825f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cde581444394ae0b342d0c1ec435c76'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.239742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1360/2741] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:00:45 2022] DELETE /identity/v3/users/5cde581444394ae0b342d0c1ec435c76 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:00:45.433338 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:45.445179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974fed8-dda6-41f1-a966-33301e07b718 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:45.505410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:45.506031 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.506302 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.506565 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] PATH_INFO: `/v3/projects/b93ff276e6894c77a231ef075b02ca9a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.507761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93ff276e6894c77a231ef075b02ca9a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:45.509692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:45.558427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93ff276e6894c77a231ef075b02ca9a', 'request_id': 'req-d63957c0-fb81-495a-9c34-62ced315e074'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.564713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93ff276e6894c77a231ef075b02ca9a', 'request_id': 'req-d63957c0-fb81-495a-9c34-62ced315e074'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.570149 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93ff276e6894c77a231ef075b02ca9a', 'request_id': 'req-d63957c0-fb81-495a-9c34-62ced315e074'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.584655 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] The token cache is being invalidate because project b93ff276e6894c77a231ef075b02ca9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:00:45.584945 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.585595 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d63957c0-fb81-495a-9c34-62ced315e074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:00:45.587082 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1382/2742] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:00:45 2022] DELETE /identity/v3/projects/b93ff276e6894c77a231ef075b02ca9a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:00:45.592207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb0c1e-8a20-4c48-839b-217b46b34c35 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.592513 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb0c1e-8a20-4c48-839b-217b46b34c35 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.592788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb0c1e-8a20-4c48-839b-217b46b34c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.612616 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d7eb0c1e-8a20-4c48-839b-217b46b34c35 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:00:45.613815 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d7eb0c1e-8a20-4c48-839b-217b46b34c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:45.675775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1361/2743] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:00:45.684897 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23ead0-61b6-41af-8794-2e90a4c48820 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.685171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23ead0-61b6-41af-8794-2e90a4c48820 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.685436 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23ead0-61b6-41af-8794-2e90a4c48820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.705980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-8f23ead0-61b6-41af-8794-2e90a4c48820 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:00:45.707194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-8f23ead0-61b6-41af-8794-2e90a4c48820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:45.716412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1383/2744] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:00:45.721009 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb0c1e-8a20-4c48-839b-217b46b34c35 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:45.730871 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb0c1e-8a20-4c48-839b-217b46b34c35 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:45.735913 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:45.736535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.736811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.737142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.737762 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 11:00:45.738057 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:45.739660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:45.751209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1362/2745] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 11:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 11:00:45.755610 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23ead0-61b6-41af-8794-2e90a4c48820 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:45.765419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23ead0-61b6-41af-8794-2e90a4c48820 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:45.770432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:45.771095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.771365 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.771653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.772295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:45.774259 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:45.816956 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1384/2746] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:00:45 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 11:00:45.821370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:45.831895 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a7364-cf06-4d4f-8d55-b3e666170641 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:45.835471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:00:45.891630 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:45.892225 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.892529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.892790 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.893441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:45.895351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:45.921798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1363/2747] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:00:45 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 11:00:45.926167 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:45.936559 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55b5e-40bd-4e29-80b4-24a22e0d77a3 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:45.942238 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:45.942801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.943116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.943477 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.944162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:45.945793 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:45.956209 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1385/2748] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:00:45.960210 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:45.971323 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eb5ae-6d96-404c-8254-1e038595487a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:45.977423 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322ed82-f294-404b-bd0d-c98907125d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:45.978013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322ed82-f294-404b-bd0d-c98907125d07 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:45.978299 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322ed82-f294-404b-bd0d-c98907125d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:45.978559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322ed82-f294-404b-bd0d-c98907125d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:45.979188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322ed82-f294-404b-bd0d-c98907125d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:45.980986 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322ed82-f294-404b-bd0d-c98907125d07 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:45.992689 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1364/2749] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:00:45.997644 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.009176 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304212-e20f-4d7e-ab33-ecd37f26dc32 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.014604 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.015295 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.015571 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.015855 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] PATH_INFO: `/v3/projects/5462c2d376344ac79d785eb7f6eeaf56/users/3a21f1f3fcce45acb150d6d254830cae/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.031541 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5462c2d376344ac79d785eb7f6eeaf56, user_id=3a21f1f3fcce45acb150d6d254830cae, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.034205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.054189 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1386/2750] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:00:45 2022] PUT /identity/v3/projects/5462c2d376344ac79d785eb7f6eeaf56/users/3a21f1f3fcce45acb150d6d254830cae/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:00:46.065011 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1229-5dc1-41ab-963e-38e21f5dd236 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.065514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1229-5dc1-41ab-963e-38e21f5dd236 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.065625 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1229-5dc1-41ab-963e-38e21f5dd236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.093973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-45cd1229-5dc1-41ab-963e-38e21f5dd236 None None] MFA Rules not processed for user `3a21f1f3fcce45acb150d6d254830cae`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:00:46.095171 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-45cd1229-5dc1-41ab-963e-38e21f5dd236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:46.165635 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1365/2751] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:00:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:00:46.170092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.190492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:00:46.235193 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e45f58-c3ef-457a-8ea2-d1111f2b7b8f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.240781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.241394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.241670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.241939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.242642 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.244618 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.273549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1387/2752] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:00:46 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 11:00:46.278196 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1229-5dc1-41ab-963e-38e21f5dd236 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.289194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1229-5dc1-41ab-963e-38e21f5dd236 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.347519 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.348155 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.348460 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.348731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.349368 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.351292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.375242 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1366/2753] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:00:46 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:00:46.380045 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.394285 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac01dd8-238d-4757-b0d3-7005478fd51f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.399612 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.400236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.400552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.400820 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.401794 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.403411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.414529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1388/2754] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:00:46.418892 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.426287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:00:46.430545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602025f5-48a8-4026-af6b-a8c759a06d82 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.437643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.438289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.438562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.438831 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] PATH_INFO: `/v3/projects/cbd5b4e62660431fae88f2a1675861ed/users/8abf7c8c412c49ac84a94c636efc7c0d/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.454103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd5b4e62660431fae88f2a1675861ed, user_id=8abf7c8c412c49ac84a94c636efc7c0d, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.456777 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.472163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1367/2755] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:00:46 2022] PUT /identity/v3/projects/cbd5b4e62660431fae88f2a1675861ed/users/8abf7c8c412c49ac84a94c636efc7c0d/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:00:46.476317 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.486564 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020186ca-a3ce-44ee-84b8-5121fa7a2919 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.544600 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303fe6-1eaa-452f-9192-ae1f6add8225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.545260 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303fe6-1eaa-452f-9192-ae1f6add8225 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.545531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303fe6-1eaa-452f-9192-ae1f6add8225 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.545795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303fe6-1eaa-452f-9192-ae1f6add8225 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.546529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df303fe6-1eaa-452f-9192-ae1f6add8225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.548293 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df303fe6-1eaa-452f-9192-ae1f6add8225 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.558314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1389/2756] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:00:46.562974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.573765 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60cbf-2def-4be1-abb0-6dc2cad714ed admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.578936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.579601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.579874 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.580137 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] PATH_INFO: `/v3/projects/cbd5b4e62660431fae88f2a1675861ed/users/8abf7c8c412c49ac84a94c636efc7c0d/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.582190 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd5b4e62660431fae88f2a1675861ed, user_id=8abf7c8c412c49ac84a94c636efc7c0d, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.584840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.595629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1368/2757] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:00:46 2022] PUT /identity/v3/projects/cbd5b4e62660431fae88f2a1675861ed/users/8abf7c8c412c49ac84a94c636efc7c0d/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:00:46.606126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541659e4-799e-4b95-8fa9-df98ec54ffff None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.606403 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541659e4-799e-4b95-8fa9-df98ec54ffff None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.606657 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541659e4-799e-4b95-8fa9-df98ec54ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.636694 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-541659e4-799e-4b95-8fa9-df98ec54ffff None None] MFA Rules not processed for user `8abf7c8c412c49ac84a94c636efc7c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:00:46.637950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-541659e4-799e-4b95-8fa9-df98ec54ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:46.711857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1390/2758] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:00:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:00:46.725135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.727156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:46.784580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108045-0d6f-4401-b4ce-a0913f3f1144 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.789256 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.789844 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.790123 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.790371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.799907 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.801399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.810678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1369/2759] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:00:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:00:46.891554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541659e4-799e-4b95-8fa9-df98ec54ffff None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:46.902277 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541659e4-799e-4b95-8fa9-df98ec54ffff None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:46.906764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584c0d7-8a1e-41d2-aba5-3f49bc0e982a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:46.907362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584c0d7-8a1e-41d2-aba5-3f49bc0e982a service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:46.908134 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584c0d7-8a1e-41d2-aba5-3f49bc0e982a service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:46.908499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584c0d7-8a1e-41d2-aba5-3f49bc0e982a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:46.919837 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:00:46.976157 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584c0d7-8a1e-41d2-aba5-3f49bc0e982a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:46.978298 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584c0d7-8a1e-41d2-aba5-3f49bc0e982a service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:46.989494 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1391/2760] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:00:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:00:49.220670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:49.222428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:49.284867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23100-c7aa-422c-9a5a-9979e1ad6cc1 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:49.289840 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531311-2bc9-484f-84c6-5a3cffd35c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:49.290456 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531311-2bc9-484f-84c6-5a3cffd35c86 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:49.290729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531311-2bc9-484f-84c6-5a3cffd35c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:49.291016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531311-2bc9-484f-84c6-5a3cffd35c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:49.293174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1e531311-2bc9-484f-84c6-5a3cffd35c86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:49.303412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531311-2bc9-484f-84c6-5a3cffd35c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:49.304880 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531311-2bc9-484f-84c6-5a3cffd35c86 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:49.313016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1370/2761] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:00:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:00:55.941650 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83fef5-d802-4837-a507-32bbdaea1b4e None None] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:55.942249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83fef5-d802-4837-a507-32bbdaea1b4e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:55.942249 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83fef5-d802-4837-a507-32bbdaea1b4e None None] PATH_INFO: `/` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:55.943508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1392/2762] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 11:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 11:00:55.950371 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ec3f-170c-45a1-a664-adafe751728d None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:55.950854 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ec3f-170c-45a1-a664-adafe751728d None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:55.950978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ec3f-170c-45a1-a664-adafe751728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:56.008115 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-7231ec3f-170c-45a1-a664-adafe751728d None None] MFA Rules not processed for user `a8f3a56366d348bfb0a5c445905dc46b`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:00:56.009408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-7231ec3f-170c-45a1-a664-adafe751728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:56.017158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1371/2763] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 11:00:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:00:56.031028 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83fef5-d802-4837-a507-32bbdaea1b4e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:56.043489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83fef5-d802-4837-a507-32bbdaea1b4e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:56.047692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f76a1e-f777-4c27-b342-f9c76afdb004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:56.048408 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f76a1e-f777-4c27-b342-f9c76afdb004 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:56.048743 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f76a1e-f777-4c27-b342-f9c76afdb004 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:56.049075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f76a1e-f777-4c27-b342-f9c76afdb004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:56.057920 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f76a1e-f777-4c27-b342-f9c76afdb004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:56.067660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f76a1e-f777-4c27-b342-f9c76afdb004 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:56.083522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1393/2764] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:00:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:00:56.168431 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5276-fa44-4800-a808-0c58fab1714b None None] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:56.168723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5276-fa44-4800-a808-0c58fab1714b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:56.169058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5276-fa44-4800-a808-0c58fab1714b None None] PATH_INFO: `/` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:56.170480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1372/2765] 10.222.0.23 () {54 vars in 921 bytes} [Fri Aug 12 11:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 11:00:56.178589 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:56.179141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:56.179420 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:56.226060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] MFA Rules not processed for user `a25194d34d67429b92944e0484de03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:00:56.227262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:00:56.297543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1394/2766] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Aug 12 11:00:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:00:56.309680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5276-fa44-4800-a808-0c58fab1714b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:00:56.320918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5276-fa44-4800-a808-0c58fab1714b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:00:56.324484 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:00:56.325092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:00:56.325351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:00:56.325684 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:00:56.336291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:00:56.337781 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:00:56.349280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1373/2767] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:00:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:06.288333 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=65 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:06.344134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:19.741667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:19.743778 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:19.762361 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe6a6f-26aa-45de-88d9-2a23d0505cd4 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:19.769075 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4470-d71f-420c-931f-a38b131157e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:19.769687 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4470-d71f-420c-931f-a38b131157e0 service nova] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:19.769962 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4470-d71f-420c-931f-a38b131157e0 service nova] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:19.770229 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4470-d71f-420c-931f-a38b131157e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:19.772781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b0bb4470-d71f-420c-931f-a38b131157e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:19.788662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4470-d71f-420c-931f-a38b131157e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:19.790536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4470-d71f-420c-931f-a38b131157e0 service nova] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:19.803219 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1395/2768] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:27.102945 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:27.171056 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76435ab-ceb7-4463-81c9-2960c75a362f service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:27.176406 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:27.177129 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:27.177298 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:27.177596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] PATH_INFO: `/v3/users/3a21f1f3fcce45acb150d6d254830cae` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:27.178791 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a21f1f3fcce45acb150d6d254830cae)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:27.180799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:27.237850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a21f1f3fcce45acb150d6d254830cae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.246967 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a21f1f3fcce45acb150d6d254830cae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.247894 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a21f1f3fcce45acb150d6d254830cae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.258432 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a21f1f3fcce45acb150d6d254830cae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.277209 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a21f1f3fcce45acb150d6d254830cae'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.286604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1374/2769] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:27 2022] DELETE /identity/v3/users/3a21f1f3fcce45acb150d6d254830cae => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:27.292633 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:27.293158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:27.293158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:27.313111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:27.314335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:27.379946 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1396/2770] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:27.390188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:27.391669 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:27.451518 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafe560-9dcf-4769-a4b8-8612a7416247 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:27.455095 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:27.455691 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:27.456064 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:27.456434 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:27.466128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:27.467559 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:27.477408 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1375/2771] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:27.771232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:27.772968 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:27.788252 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb1431-f041-4c45-9ab8-9fbc57c66c97 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:27.793801 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:27.794505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:27.794870 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:27.794986 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] PATH_INFO: `/v3/projects/5462c2d376344ac79d785eb7f6eeaf56` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:27.796201 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5462c2d376344ac79d785eb7f6eeaf56)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:27.798142 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:27.854164 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5462c2d376344ac79d785eb7f6eeaf56', 'request_id': 'req-068d8aed-9864-4d21-addc-b72e3a9c275a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.862843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5462c2d376344ac79d785eb7f6eeaf56', 'request_id': 'req-068d8aed-9864-4d21-addc-b72e3a9c275a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.869550 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5462c2d376344ac79d785eb7f6eeaf56', 'request_id': 'req-068d8aed-9864-4d21-addc-b72e3a9c275a'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.884905 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] The token cache is being invalidate because project 5462c2d376344ac79d785eb7f6eeaf56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:27.885200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.885885 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:27.887435 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1397/2772] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:27 2022] DELETE /identity/v3/projects/5462c2d376344ac79d785eb7f6eeaf56 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:27.891602 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:27.902268 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960235f-4605-48b2-a676-ba3928f6f60b service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:27.920893 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:27.962819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:27.963457 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:27.963723 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:27.963984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] PATH_INFO: `/v3/users/8abf7c8c412c49ac84a94c636efc7c0d` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:27.965317 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8abf7c8c412c49ac84a94c636efc7c0d)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:27.967313 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:28.021820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8abf7c8c412c49ac84a94c636efc7c0d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.029136 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8abf7c8c412c49ac84a94c636efc7c0d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.029806 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8abf7c8c412c49ac84a94c636efc7c0d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.039775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8abf7c8c412c49ac84a94c636efc7c0d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.052007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-73cfdef9-cb1e-4118-9f4b-c029af96d76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8abf7c8c412c49ac84a94c636efc7c0d'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.057842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1376/2773] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:27 2022] DELETE /identity/v3/users/8abf7c8c412c49ac84a94c636efc7c0d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:28.481862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:28.483609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:28.497328 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d8aed-9864-4d21-addc-b72e3a9c275a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:28.568412 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:28.568964 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:28.569236 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:28.569501 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] PATH_INFO: `/v3/projects/cbd5b4e62660431fae88f2a1675861ed` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:28.570609 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd5b4e62660431fae88f2a1675861ed)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:28.572663 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:28.585319 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:28.626134 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd5b4e62660431fae88f2a1675861ed', 'request_id': 'req-3e8bba26-1a19-4a3b-8232-17d37628e31b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.633430 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd5b4e62660431fae88f2a1675861ed', 'request_id': 'req-3e8bba26-1a19-4a3b-8232-17d37628e31b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.640520 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd5b4e62660431fae88f2a1675861ed', 'request_id': 'req-3e8bba26-1a19-4a3b-8232-17d37628e31b'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.656386 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] The token cache is being invalidate because project cbd5b4e62660431fae88f2a1675861ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:28.656684 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.657326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3e8bba26-1a19-4a3b-8232-17d37628e31b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:28.658860 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1398/2774] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:28 2022] DELETE /identity/v3/projects/cbd5b4e62660431fae88f2a1675861ed => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:28.664305 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fca0f-29e8-469a-ac10-429dd7e0073b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:28.664855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fca0f-29e8-469a-ac10-429dd7e0073b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:28.664855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fca0f-29e8-469a-ac10-429dd7e0073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:28.686971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-bb6fca0f-29e8-469a-ac10-429dd7e0073b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:28.688245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-bb6fca0f-29e8-469a-ac10-429dd7e0073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:28.759132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1377/2775] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:28.767532 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1997e-f99a-4dd6-8f11-5b12247d6a7d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:28.767809 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1997e-f99a-4dd6-8f11-5b12247d6a7d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:28.768081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1997e-f99a-4dd6-8f11-5b12247d6a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:28.789660 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5ea1997e-f99a-4dd6-8f11-5b12247d6a7d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:28.790904 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5ea1997e-f99a-4dd6-8f11-5b12247d6a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:28.799447 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1399/2776] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:28.804119 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fca0f-29e8-469a-ac10-429dd7e0073b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:28.815094 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fca0f-29e8-469a-ac10-429dd7e0073b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:28.821662 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:28.822311 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:28.822645 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:28.823069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:28.823757 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 11:01:28.824044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:28.825732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:28.843370 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1378/2777] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 11:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 11:01:28.848822 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1997e-f99a-4dd6-8f11-5b12247d6a7d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:28.860707 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1997e-f99a-4dd6-8f11-5b12247d6a7d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:28.866358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:28.866980 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:28.867278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:28.867542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:28.868182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:28.870132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:28.906839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1400/2778] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:01:28 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:28.913698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:28.922807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d950b8-820c-49fb-bc8a-4905d43eec53 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:28.958146 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:28.998857 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:28.999474 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:28.999742 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.000038 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.000705 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.002731 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.029729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1379/2779] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:01:28 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:29.034179 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.046145 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06617d-e81e-4d78-8b9a-291211ecc93f admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.051915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.052567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.052863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.053110 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.053768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.055432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.067235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1401/2780] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:29.071838 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.084103 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4837cb-88aa-4b7c-960f-afeab143e316 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.090013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511e9b-d9a2-4bdf-9d33-153f827f4445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.090732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511e9b-d9a2-4bdf-9d33-153f827f4445 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.091006 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511e9b-d9a2-4bdf-9d33-153f827f4445 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.091276 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511e9b-d9a2-4bdf-9d33-153f827f4445 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.091914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511e9b-d9a2-4bdf-9d33-153f827f4445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.093653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511e9b-d9a2-4bdf-9d33-153f827f4445 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.106996 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1380/2781] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:29.111375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.122670 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a962e17-2d38-4361-a746-01b3c9076f21 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.129326 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.130077 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.130470 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.130850 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] PATH_INFO: `/v3/projects/872b7531d4154851a49302016dc8bb7a/users/043e0f7f695a4f03bb1c24a56c969fd7/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.151039 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872b7531d4154851a49302016dc8bb7a, user_id=043e0f7f695a4f03bb1c24a56c969fd7, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.154387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.170383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1402/2782] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:01:29 2022] PUT /identity/v3/projects/872b7531d4154851a49302016dc8bb7a/users/043e0f7f695a4f03bb1c24a56c969fd7/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:29.183667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.184290 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.184920 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.222555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] MFA Rules not processed for user `043e0f7f695a4f03bb1c24a56c969fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:29.224529 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:29.304138 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1381/2783] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:29.310321 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.405204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ae558-7a65-41a1-9f65-a414b9e52146 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.410988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.411659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.411927 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.412193 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.412866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.413601 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.415619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:29.416372 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.463958 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1403/2784] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:01:29 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:29.468752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.519944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b57341-7486-4cdd-ab83-f7d43d81a8e8 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.535545 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:29.566902 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:29.622965 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a25194d34d67429b92944e0484de03cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.623939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.624324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.624661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.625888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0d7bd-3c09-48de-9166-b607e072dd51 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.628335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:29.638868 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.640522 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.641594 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.643542 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.645309 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.645466 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.647044 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.648418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.663014 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1382/2785] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:29.685596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1404/2786] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:01:29 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:29.689947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.701698 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa54e4-8414-4fe2-b603-f38701b1ec3f service nova] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.707597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.708335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.708678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.709021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.709729 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.711682 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.728974 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1383/2787] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:29.730495 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.742103 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae6ed-91f3-4d86-8029-d8adb9b1255a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.747878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.748597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.748857 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.749120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] PATH_INFO: `/v3/projects/ad00c230eee84217b7ae8a2cdf120347/users/b578508fff55451e8bfd476c6706d20e/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.767726 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad00c230eee84217b7ae8a2cdf120347, user_id=b578508fff55451e8bfd476c6706d20e, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.770967 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.790988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1405/2788] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:01:29 2022] PUT /identity/v3/projects/ad00c230eee84217b7ae8a2cdf120347/users/b578508fff55451e8bfd476c6706d20e/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:29.795366 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.806292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213d3e4-0f3e-45e8-9319-d63a16a7b146 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.871973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a91f6-aad1-45fa-8e79-5fcbfdbf0566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.872540 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a91f6-aad1-45fa-8e79-5fcbfdbf0566 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.872811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a91f6-aad1-45fa-8e79-5fcbfdbf0566 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.873078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a91f6-aad1-45fa-8e79-5fcbfdbf0566 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.873706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a91f6-aad1-45fa-8e79-5fcbfdbf0566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.875296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a91f6-aad1-45fa-8e79-5fcbfdbf0566 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.888316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1384/2789] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:29.890449 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:29.901279 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5db47d-41aa-4548-943b-95516b2afe9b admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:29.906839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:29.907499 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.907768 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.908032 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] PATH_INFO: `/v3/projects/ad00c230eee84217b7ae8a2cdf120347/users/b578508fff55451e8bfd476c6706d20e/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.910278 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad00c230eee84217b7ae8a2cdf120347, user_id=b578508fff55451e8bfd476c6706d20e, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:29.912954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:29.924301 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1406/2790] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:01:29 2022] PUT /identity/v3/projects/ad00c230eee84217b7ae8a2cdf120347/users/b578508fff55451e8bfd476c6706d20e/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:29.934715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc503c-6215-43dd-b157-02ad8838ebfb None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:29.934997 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc503c-6215-43dd-b157-02ad8838ebfb None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:29.935262 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc503c-6215-43dd-b157-02ad8838ebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:29.964017 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-19bc503c-6215-43dd-b157-02ad8838ebfb None None] MFA Rules not processed for user `b578508fff55451e8bfd476c6706d20e`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:29.965291 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-19bc503c-6215-43dd-b157-02ad8838ebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:30.042058 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1385/2791] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:30.054241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:30.056044 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:30.117150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b515f-a551-49eb-b870-ef4deefc6c65 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:30.121591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:30.122410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:30.122524 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:30.122713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:30.132710 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:30.134359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:30.144394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1407/2792] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:30.230008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc503c-6215-43dd-b157-02ad8838ebfb None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:30.240643 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc503c-6215-43dd-b157-02ad8838ebfb None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:30.244629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8bf8d-576e-4646-8520-e44c0934347e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:30.245231 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8bf8d-576e-4646-8520-e44c0934347e service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:30.245502 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8bf8d-576e-4646-8520-e44c0934347e service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:30.245768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8bf8d-576e-4646-8520-e44c0934347e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:30.277260 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=100 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:30.306322 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8bf8d-576e-4646-8520-e44c0934347e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:30.307798 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8bf8d-576e-4646-8520-e44c0934347e service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:30.317282 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1386/2793] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:39.314120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:39.380379 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3694db7-935c-4e2c-9bb6-da34d94c9fd7 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:39.385383 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:39.386033 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:39.386314 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:39.386587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] PATH_INFO: `/v3/users/043e0f7f695a4f03bb1c24a56c969fd7` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:39.387701 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=043e0f7f695a4f03bb1c24a56c969fd7)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:39.389764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:39.414109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:39.451831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043e0f7f695a4f03bb1c24a56c969fd7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:39.452579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043e0f7f695a4f03bb1c24a56c969fd7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:39.453270 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043e0f7f695a4f03bb1c24a56c969fd7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:39.460141 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043e0f7f695a4f03bb1c24a56c969fd7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:39.469100 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043e0f7f695a4f03bb1c24a56c969fd7'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:39.486410 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1408/2794] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:39 2022] DELETE /identity/v3/users/043e0f7f695a4f03bb1c24a56c969fd7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:39.491101 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:39.491638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:39.491638 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:39.512940 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:39.514132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:39.580928 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:39.582602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:39.589818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1387/2795] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:39.599342 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:39.600852 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:39.642518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379b2b4-bb54-4701-9045-184f816cc8e5 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:39.646518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:39.647132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:39.647407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:39.647675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:39.649614 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:39.659891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1f697-b682-4b17-91a6-2dd87b1e2f5c None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:39.663224 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:39.664156 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:39.664713 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:39.664833 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:39.665059 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:39.665340 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:39.674499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:39.674915 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1409/2796] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 11:01:39.675916 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:39.685866 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1388/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 11:01:40.146760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:40.148356 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:40.162027 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d148e9a-1f58-4d05-8c13-ba8ded1b3264 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:40.166825 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:40.167456 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:40.167725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:40.167990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] PATH_INFO: `/v3/projects/872b7531d4154851a49302016dc8bb7a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:40.169128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=872b7531d4154851a49302016dc8bb7a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:40.171841 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:40.223829 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872b7531d4154851a49302016dc8bb7a', 'request_id': 'req-6f9f6adf-e353-4eec-9145-7cd284a5b6af'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.230108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872b7531d4154851a49302016dc8bb7a', 'request_id': 'req-6f9f6adf-e353-4eec-9145-7cd284a5b6af'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.235625 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872b7531d4154851a49302016dc8bb7a', 'request_id': 'req-6f9f6adf-e353-4eec-9145-7cd284a5b6af'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.249752 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] The token cache is being invalidate because project 872b7531d4154851a49302016dc8bb7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:40.250047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.250779 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.256180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1410/2798] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:40 2022] DELETE /identity/v3/projects/872b7531d4154851a49302016dc8bb7a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:40.257354 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:40.270972 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9a706-9f88-4ff3-9233-8c856922a2b5 service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:40.327150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:40.327805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:40.328065 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:40.328388 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] PATH_INFO: `/v3/users/b578508fff55451e8bfd476c6706d20e` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:40.329496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b578508fff55451e8bfd476c6706d20e)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:40.331471 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:40.346617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:40.382492 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b578508fff55451e8bfd476c6706d20e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.389504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b578508fff55451e8bfd476c6706d20e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.390165 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b578508fff55451e8bfd476c6706d20e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.399464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b578508fff55451e8bfd476c6706d20e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.412178 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-712e4e40-c7dd-4d0b-b8ce-984ea115bf13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b578508fff55451e8bfd476c6706d20e'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.418656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1389/2799] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:40 2022] DELETE /identity/v3/users/b578508fff55451e8bfd476c6706d20e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:40.710725 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:40.712649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:40.728128 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f6adf-e353-4eec-9145-7cd284a5b6af admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:40.750012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:40.794847 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:40.795505 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:40.795960 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:40.796405 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] PATH_INFO: `/v3/projects/ad00c230eee84217b7ae8a2cdf120347` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:40.797416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad00c230eee84217b7ae8a2cdf120347)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:40.799342 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:40.858783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad00c230eee84217b7ae8a2cdf120347', 'request_id': 'req-28ce347b-2322-4722-9b83-d02bb42e8fc3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.865318 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad00c230eee84217b7ae8a2cdf120347', 'request_id': 'req-28ce347b-2322-4722-9b83-d02bb42e8fc3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.871451 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad00c230eee84217b7ae8a2cdf120347', 'request_id': 'req-28ce347b-2322-4722-9b83-d02bb42e8fc3'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.886111 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] The token cache is being invalidate because project ad00c230eee84217b7ae8a2cdf120347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:40.886574 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.887305 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-28ce347b-2322-4722-9b83-d02bb42e8fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:40.889022 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1411/2800] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:40 2022] DELETE /identity/v3/projects/ad00c230eee84217b7ae8a2cdf120347 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:40.894008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2f882-09c8-4b0e-a73c-9b0bddb642e4 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:40.894512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2f882-09c8-4b0e-a73c-9b0bddb642e4 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:40.894610 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2f882-09c8-4b0e-a73c-9b0bddb642e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:40.915329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-6be2f882-09c8-4b0e-a73c-9b0bddb642e4 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:40.916888 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-6be2f882-09c8-4b0e-a73c-9b0bddb642e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:40.975951 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1390/2801] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 11:01:40.985062 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d51c1b-ba9e-4a60-99e9-071bd474152d None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:40.985416 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d51c1b-ba9e-4a60-99e9-071bd474152d None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:40.985754 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d51c1b-ba9e-4a60-99e9-071bd474152d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.010358 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-76d51c1b-ba9e-4a60-99e9-071bd474152d None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:41.011583 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-76d51c1b-ba9e-4a60-99e9-071bd474152d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:41.020488 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1412/2802] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:41.023785 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2f882-09c8-4b0e-a73c-9b0bddb642e4 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.034337 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2f882-09c8-4b0e-a73c-9b0bddb642e4 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.039609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.040376 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.040740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.041020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.041670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 11:01:41.042034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.043660 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.056008 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1391/2803] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 11:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 11:01:41.060758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d51c1b-ba9e-4a60-99e9-071bd474152d None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.070375 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d51c1b-ba9e-4a60-99e9-071bd474152d None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.076756 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.077370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.077664 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.077939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.078518 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.080489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.111861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1413/2804] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:01:41 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:41.116075 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.127343 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826e0f0-31ec-42e3-89cf-b84e013e2643 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.191464 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.192070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.192491 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.192617 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.194619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.195802 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.222711 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1392/2805] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:01:41 2022] POST /identity/v3/users => generated 278 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:41.227030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.241481 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70aa4e-3f5e-4c0a-b26e-cc23ffdbd4af admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.248168 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.248845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.249159 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.249433 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.250085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.251878 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.266874 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1414/2806] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:41.270402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.282618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf2b63-88f0-4718-b9e7-97d617077590 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.288128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ba29b-f03a-4285-87b1-7d9b699c8173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.288796 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ba29b-f03a-4285-87b1-7d9b699c8173 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.289069 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ba29b-f03a-4285-87b1-7d9b699c8173 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.289566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ba29b-f03a-4285-87b1-7d9b699c8173 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.290004 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ba29b-f03a-4285-87b1-7d9b699c8173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.291925 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ba29b-f03a-4285-87b1-7d9b699c8173 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.295212 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:41.310615 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.312653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1393/2807] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:41.321755 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5675-7045-497e-8a7c-8b20712ecde4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.326893 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.327587 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.327864 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.328135 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] PATH_INFO: `/v3/projects/dc0287169cd74d70846c3953580a8a7c/users/928bd80c1e034945b3dcfc3f96fc084b/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.358325 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0287169cd74d70846c3953580a8a7c, user_id=928bd80c1e034945b3dcfc3f96fc084b, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.361802 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.403838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1415/2808] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:01:41 2022] PUT /identity/v3/projects/dc0287169cd74d70846c3953580a8a7c/users/928bd80c1e034945b3dcfc3f96fc084b/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 11:01:41.477139 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75865160-54aa-40a3-9116-8a296825dd14 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.477482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75865160-54aa-40a3-9116-8a296825dd14 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.477768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75865160-54aa-40a3-9116-8a296825dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.510494 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-75865160-54aa-40a3-9116-8a296825dd14 None None] MFA Rules not processed for user `928bd80c1e034945b3dcfc3f96fc084b`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:41.511054 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-75865160-54aa-40a3-9116-8a296825dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:41.594623 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1394/2809] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:41.599171 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.658109 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:41.675130 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce397da-09a5-486b-b472-873487484b0c admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.680064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.680981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.681123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.681391 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.682024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.683935 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.712978 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1416/2810] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:01:41 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:41.717013 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75865160-54aa-40a3-9116-8a296825dd14 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.729609 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75865160-54aa-40a3-9116-8a296825dd14 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.795071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.795805 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.796206 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.796568 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.797307 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.799331 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.823113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1395/2811] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:01:41 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:41.827241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.839478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a4538-56ae-47c7-956b-c22e5370f876 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.844407 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.845014 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.845299 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.845567 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.846215 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.847882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.860160 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1417/2812] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:41.864403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.875736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d88af-307a-4df8-8eeb-f8409573f30e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:41.881277 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:41.882747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:41.882747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:41.882747 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] PATH_INFO: `/v3/projects/672033a694244dbfa565aede01549a48/users/114f267fdbdb44c1af70f1b4d55f4b83/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:41.905955 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=672033a694244dbfa565aede01549a48, user_id=114f267fdbdb44c1af70f1b4d55f4b83, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:41.909316 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:41.924717 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1396/2813] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:01:41 2022] PUT /identity/v3/projects/672033a694244dbfa565aede01549a48/users/114f267fdbdb44c1af70f1b4d55f4b83/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:41.929071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:41.940732 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e34d-3fb6-41da-819a-f9951d9708ca admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:42.001434 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ded29-8939-4df8-ac77-3a89b80cf74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:42.002080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ded29-8939-4df8-ac77-3a89b80cf74e admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:42.002343 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ded29-8939-4df8-ac77-3a89b80cf74e admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:42.002766 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ded29-8939-4df8-ac77-3a89b80cf74e admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:42.003406 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ded29-8939-4df8-ac77-3a89b80cf74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:42.005090 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ded29-8939-4df8-ac77-3a89b80cf74e admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:42.016579 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1418/2814] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:42.020872 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:42.032091 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee0792-9c57-4fda-959f-89c90d2eaa9f admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:42.036984 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:42.037732 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:42.038068 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:42.038401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] PATH_INFO: `/v3/projects/672033a694244dbfa565aede01549a48/users/114f267fdbdb44c1af70f1b4d55f4b83/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:42.040788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672033a694244dbfa565aede01549a48, user_id=114f267fdbdb44c1af70f1b4d55f4b83, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:42.043429 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:42.055266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1397/2815] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:01:42 2022] PUT /identity/v3/projects/672033a694244dbfa565aede01549a48/users/114f267fdbdb44c1af70f1b4d55f4b83/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:42.065552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d335-9d8d-4a9e-89ea-d426fdc96533 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:42.065832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d335-9d8d-4a9e-89ea-d426fdc96533 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:42.066101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003d335-9d8d-4a9e-89ea-d426fdc96533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:42.095015 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-7003d335-9d8d-4a9e-89ea-d426fdc96533 None None] MFA Rules not processed for user `114f267fdbdb44c1af70f1b4d55f4b83`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:42.096132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-7003d335-9d8d-4a9e-89ea-d426fdc96533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:42.168998 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1419/2816] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:42.182375 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:42.184655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:42.240706 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:42.261507 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0cca3-43e5-4cee-899f-7e6227ceb515 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:42.265619 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:42.266441 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:42.266701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:42.266960 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:42.277954 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:42.279437 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:42.295102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1398/2817] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:42.393871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003d335-9d8d-4a9e-89ea-d426fdc96533 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:42.406546 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003d335-9d8d-4a9e-89ea-d426fdc96533 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:42.411329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:42.412012 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:42.412374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:42.412858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:42.474688 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:42.490899 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:42.492464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:42.505085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1420/2818] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:46.449973 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:46.451961 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:46.520106 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f450d-3703-4545-9c63-8c78750d22af service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:46.525541 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:46.526180 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:46.526452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:46.526721 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] PATH_INFO: `/v3/users/922f49fa7d324ebdade043e1be1a476f` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:46.528329 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=922f49fa7d324ebdade043e1be1a476f)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:46.530274 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:46.583971 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922f49fa7d324ebdade043e1be1a476f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.590745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922f49fa7d324ebdade043e1be1a476f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.591428 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922f49fa7d324ebdade043e1be1a476f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.599739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922f49fa7d324ebdade043e1be1a476f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.612403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922f49fa7d324ebdade043e1be1a476f'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.618763 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1399/2819] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:46 2022] DELETE /identity/v3/users/922f49fa7d324ebdade043e1be1a476f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:46.798750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:46.812287 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d4de8-0198-4c77-bafc-d5337d3030e7 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:46.877205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:46.877891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:46.878158 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:46.878423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] PATH_INFO: `/v3/projects/89943304402242278f5614ea5bde6092` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:46.879580 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89943304402242278f5614ea5bde6092)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:46.881596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:46.934795 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89943304402242278f5614ea5bde6092', 'request_id': 'req-181a5898-18b4-4759-8d07-4fc5c3e85956'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.941114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89943304402242278f5614ea5bde6092', 'request_id': 'req-181a5898-18b4-4759-8d07-4fc5c3e85956'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.946497 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89943304402242278f5614ea5bde6092', 'request_id': 'req-181a5898-18b4-4759-8d07-4fc5c3e85956'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.959807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] The token cache is being invalidate because project 89943304402242278f5614ea5bde6092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:46.960095 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.960764 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:46.962320 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1421/2820] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:46 2022] DELETE /identity/v3/projects/89943304402242278f5614ea5bde6092 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:46.966549 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:46.968005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:46.981715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92491adc-6e36-4902-bc5f-1a90e8d99a2a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:47.043211 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:47.043942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:47.044232 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:47.044528 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] PATH_INFO: `/v3/users/683fe550384b4a31ba0e795fda8818ed` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:47.045764 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] RBAC: Authorizing `identity:delete_user(user_id=683fe550384b4a31ba0e795fda8818ed)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:47.047789 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:47.050473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:47.097891 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683fe550384b4a31ba0e795fda8818ed'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.104710 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683fe550384b4a31ba0e795fda8818ed'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.105597 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683fe550384b4a31ba0e795fda8818ed'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.115646 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683fe550384b4a31ba0e795fda8818ed'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.127734 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683fe550384b4a31ba0e795fda8818ed'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.133767 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1400/2821] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:46 2022] DELETE /identity/v3/users/683fe550384b4a31ba0e795fda8818ed => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:47.324322 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:47.325780 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:47.340013 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a5898-18b4-4759-8d07-4fc5c3e85956 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:47.402862 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:47.403492 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:47.403761 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:47.404049 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] PATH_INFO: `/v3/projects/a61f3df50a194d01bc1cf39a9963ba2a` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:47.405200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61f3df50a194d01bc1cf39a9963ba2a)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:47.407127 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:47.423838 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:47.457509 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61f3df50a194d01bc1cf39a9963ba2a', 'request_id': 'req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.463781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61f3df50a194d01bc1cf39a9963ba2a', 'request_id': 'req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.470085 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61f3df50a194d01bc1cf39a9963ba2a', 'request_id': 'req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.491569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] The token cache is being invalidate because project a61f3df50a194d01bc1cf39a9963ba2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:47.491863 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.492718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6cb31bc8-2b63-47a9-8199-2d5ad6c40d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:47.495099 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1422/2822] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:47 2022] DELETE /identity/v3/projects/a61f3df50a194d01bc1cf39a9963ba2a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:57.130596 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:57.486151 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:57.728510 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:57.729910 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:57.743545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5e136-ef90-4dfb-841d-711176e6230d admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:57.803599 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:57.804216 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:57.804514 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:57.804776 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] PATH_INFO: `/v3/users/928bd80c1e034945b3dcfc3f96fc084b` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:57.805933 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=928bd80c1e034945b3dcfc3f96fc084b)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:57.807850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:57.856379 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '928bd80c1e034945b3dcfc3f96fc084b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:57.863034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '928bd80c1e034945b3dcfc3f96fc084b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:57.863677 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '928bd80c1e034945b3dcfc3f96fc084b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:57.872233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '928bd80c1e034945b3dcfc3f96fc084b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:57.884071 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '928bd80c1e034945b3dcfc3f96fc084b'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:57.889991 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1401/2823] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:57 2022] DELETE /identity/v3/users/928bd80c1e034945b3dcfc3f96fc084b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:57.895498 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:57.895858 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:57.896233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:57.925105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:57.926871 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:58.004667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1423/2824] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:58.015235 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:58.016862 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:58.078970 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c7859-a48f-4502-9375-9e756909f057 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:58.082851 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:58.083461 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:58.083715 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:58.083979 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:58.093060 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:58.094487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:58.103604 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1402/2825] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:01:58.390760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:58.392241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:58.408195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14999630-aaaf-4b9b-8810-259d8c90cb87 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:58.413628 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:58.421057 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:58.421344 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:58.421629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] PATH_INFO: `/v3/projects/dc0287169cd74d70846c3953580a8a7c` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:58.422844 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0287169cd74d70846c3953580a8a7c)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:58.424810 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:58.472925 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0287169cd74d70846c3953580a8a7c', 'request_id': 'req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.478966 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0287169cd74d70846c3953580a8a7c', 'request_id': 'req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.484703 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0287169cd74d70846c3953580a8a7c', 'request_id': 'req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.500040 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] The token cache is being invalidate because project dc0287169cd74d70846c3953580a8a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:58.500649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.500988 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.502444 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1424/2826] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:58 2022] DELETE /identity/v3/projects/dc0287169cd74d70846c3953580a8a7c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:58.507207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:58.519855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b9b21-01d4-4564-9cb4-30f315f9bcac service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:58.542956 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:58.593583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:58.594219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:58.594493 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:58.594761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] PATH_INFO: `/v3/users/114f267fdbdb44c1af70f1b4d55f4b83` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:58.596020 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=114f267fdbdb44c1af70f1b4d55f4b83)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:58.597998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:58.656824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114f267fdbdb44c1af70f1b4d55f4b83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.668132 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114f267fdbdb44c1af70f1b4d55f4b83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.669073 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114f267fdbdb44c1af70f1b4d55f4b83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.678184 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114f267fdbdb44c1af70f1b4d55f4b83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.692402 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-ff1ea7b3-2aa0-4234-b247-cd3c226cfb0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114f267fdbdb44c1af70f1b4d55f4b83'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:58.699556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1403/2827] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:01:58 2022] DELETE /identity/v3/users/114f267fdbdb44c1af70f1b4d55f4b83 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:58.996879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:58.998464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:59.013626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a647d6-b4dc-4f73-8f5f-3336aac1e526 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.072404 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.072937 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.073208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.073469 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] PATH_INFO: `/v3/projects/672033a694244dbfa565aede01549a48` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.074570 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672033a694244dbfa565aede01549a48)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.076493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.111845 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:59.130288 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672033a694244dbfa565aede01549a48', 'request_id': 'req-266c4df0-6626-4540-9200-760af44f4338'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:59.136592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672033a694244dbfa565aede01549a48', 'request_id': 'req-266c4df0-6626-4540-9200-760af44f4338'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:59.142554 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672033a694244dbfa565aede01549a48', 'request_id': 'req-266c4df0-6626-4540-9200-760af44f4338'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:59.157591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] The token cache is being invalidate because project 672033a694244dbfa565aede01549a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:01:59.157879 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:59.158515 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-266c4df0-6626-4540-9200-760af44f4338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:01:59.160004 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1425/2828] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:01:58 2022] DELETE /identity/v3/projects/672033a694244dbfa565aede01549a48 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:59.165524 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568f17-2251-4de9-be1c-a538415c8278 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.165818 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568f17-2251-4de9-be1c-a538415c8278 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.166089 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04568f17-2251-4de9-be1c-a538415c8278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.187473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-04568f17-2251-4de9-be1c-a538415c8278 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:59.188722 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-04568f17-2251-4de9-be1c-a538415c8278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:59.250694 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1404/2829] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:59.258749 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f097-04c0-4f18-a102-786f5393d165 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.259030 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f097-04c0-4f18-a102-786f5393d165 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.259291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250f097-04c0-4f18-a102-786f5393d165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.280493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-0250f097-04c0-4f18-a102-786f5393d165 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:59.281685 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-0250f097-04c0-4f18-a102-786f5393d165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:59.293762 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1426/2830] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:01:59.299860 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04568f17-2251-4de9-be1c-a538415c8278 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.310078 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04568f17-2251-4de9-be1c-a538415c8278 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.315444 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.316021 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.316286 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.316580 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] PATH_INFO: `/v3/domains` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.317215 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 11:01:59.317530 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.319110 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.331280 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1405/2831] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 11:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 11:01:59.335689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250f097-04c0-4f18-a102-786f5393d165 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.345758 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250f097-04c0-4f18-a102-786f5393d165 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.351549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.352200 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.352513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.352782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.353441 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.355354 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.385212 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1427/2832] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:01:59 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:59.389608 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.399758 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe4199-221b-4964-bcdf-6dc4b218057e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.426957 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:01:59.468614 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.469401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.469739 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.470076 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.471042 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.474162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.506556 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1406/2833] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:01:59 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:59.511114 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.526271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704837a-9069-4540-b3ce-0a6d22d5782c admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.532248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.532888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.533196 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.533467 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.534116 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.536578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.550734 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1428/2834] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 11:01:59.555099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.566015 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8b9e4-d4ee-4f65-8167-6511a67279cb admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.571150 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82e2f0-50ef-45bd-96c4-49ccb19e6f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.571743 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82e2f0-50ef-45bd-96c4-49ccb19e6f2e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.572016 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82e2f0-50ef-45bd-96c4-49ccb19e6f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.572439 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82e2f0-50ef-45bd-96c4-49ccb19e6f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.573257 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82e2f0-50ef-45bd-96c4-49ccb19e6f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.575538 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82e2f0-50ef-45bd-96c4-49ccb19e6f2e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.588811 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1407/2835] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:01:59.594143 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.605214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc25683-2429-4977-86cd-5eb191b8ca0a admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.610478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.611113 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.611384 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.611647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] PATH_INFO: `/v3/projects/43210c40bca84b519484faaa25e07875/users/1df5b47d15d1426ab3c0ee0bbee6d6d4/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.626592 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=43210c40bca84b519484faaa25e07875, user_id=1df5b47d15d1426ab3c0ee0bbee6d6d4, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.629281 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.644646 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1429/2836] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:01:59 2022] PUT /identity/v3/projects/43210c40bca84b519484faaa25e07875/users/1df5b47d15d1426ab3c0ee0bbee6d6d4/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:01:59.656135 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef84aa-8479-47e8-bca3-7df80a622392 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.656426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef84aa-8479-47e8-bca3-7df80a622392 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.656692 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef84aa-8479-47e8-bca3-7df80a622392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.684536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-b9ef84aa-8479-47e8-bca3-7df80a622392 None None] MFA Rules not processed for user `1df5b47d15d1426ab3c0ee0bbee6d6d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:01:59.685718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b9ef84aa-8479-47e8-bca3-7df80a622392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:01:59.748509 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1408/2837] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:01:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 11:01:59.753292 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.817888 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78edaa0-bece-47ae-8e18-ba800889c3dc admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.823038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.823637 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.823909 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.824162 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.824910 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.826789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.859377 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1430/2838] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:01:59 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:01:59.863918 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef84aa-8479-47e8-bca3-7df80a622392 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.878292 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef84aa-8479-47e8-bca3-7df80a622392 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.937450 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.938034 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.938302 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.938562 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] PATH_INFO: `/v3/users` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.939189 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.941122 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.964566 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1409/2839] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:01:59 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 11:01:59.968569 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:01:59.979576 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffbb6e-17fb-457a-aa53-2b640f2d70f4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:01:59.984369 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:01:59.984957 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:01:59.985226 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:01:59.985485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:01:59.986119 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:01:59.987791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:01:59.998536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1431/2840] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:02:00.003199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:00.014144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b952954-7d1b-4bfa-9cf8-ec54868cbf15 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:00.019867 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:00.020579 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.020850 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.021113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] PATH_INFO: `/v3/projects/4651f0649d44421aba9e71ef7b698deb/users/156024649ac04d8b9269b353f8dcc928/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.037498 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4651f0649d44421aba9e71ef7b698deb, user_id=156024649ac04d8b9269b353f8dcc928, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:00.040142 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:00.057724 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1410/2841] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:02:00 2022] PUT /identity/v3/projects/4651f0649d44421aba9e71ef7b698deb/users/156024649ac04d8b9269b353f8dcc928/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:00.062105 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:00.072508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dddee-6ee7-47fd-9eac-8898a758a7c6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:00.093716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:00.135673 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec68fb-4ee0-419c-a820-b9070c8725a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:00.136190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec68fb-4ee0-419c-a820-b9070c8725a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.136500 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec68fb-4ee0-419c-a820-b9070c8725a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.136727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec68fb-4ee0-419c-a820-b9070c8725a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.137394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec68fb-4ee0-419c-a820-b9070c8725a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:00.138993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec68fb-4ee0-419c-a820-b9070c8725a2 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:00.149558 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1432/2842] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:02:00.154378 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:00.160147 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:00.165842 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5a373-4e30-488f-a917-818ffdac6a39 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:00.171661 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:00.172357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.172634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.172900 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] PATH_INFO: `/v3/projects/4651f0649d44421aba9e71ef7b698deb/users/156024649ac04d8b9269b353f8dcc928/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.174804 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4651f0649d44421aba9e71ef7b698deb, user_id=156024649ac04d8b9269b353f8dcc928, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:00.177480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:00.188488 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1411/2843] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:02:00 2022] PUT /identity/v3/projects/4651f0649d44421aba9e71ef7b698deb/users/156024649ac04d8b9269b353f8dcc928/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:00.200218 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3816d-0215-4d33-a034-7d4e1e933ce6 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.200552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3816d-0215-4d33-a034-7d4e1e933ce6 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.200821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3816d-0215-4d33-a034-7d4e1e933ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.230466 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-72d3816d-0215-4d33-a034-7d4e1e933ce6 None None] MFA Rules not processed for user `156024649ac04d8b9269b353f8dcc928`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:00.231667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-72d3816d-0215-4d33-a034-7d4e1e933ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:00.312641 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1433/2844] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:00.324454 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:00.326473 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:00.387153 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56a376-0722-4c96-86d4-2b2da5d33187 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:00.391249 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb303452-4c07-4c96-897e-8746f07fe37e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:00.391847 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb303452-4c07-4c96-897e-8746f07fe37e service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.392121 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb303452-4c07-4c96-897e-8746f07fe37e service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.392426 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb303452-4c07-4c96-897e-8746f07fe37e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.403394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb303452-4c07-4c96-897e-8746f07fe37e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:00.404919 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb303452-4c07-4c96-897e-8746f07fe37e service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:00.417158 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1412/2845] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:02:00.492723 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3816d-0215-4d33-a034-7d4e1e933ce6 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:00.561783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3816d-0215-4d33-a034-7d4e1e933ce6 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:00.566963 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:00.567586 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.567859 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.568123 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] PATH_INFO: `/v3/users/1df5b47d15d1426ab3c0ee0bbee6d6d4` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.569291 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df5b47d15d1426ab3c0ee0bbee6d6d4)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:00.571208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:00.623626 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df5b47d15d1426ab3c0ee0bbee6d6d4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:00.624388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df5b47d15d1426ab3c0ee0bbee6d6d4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:00.624995 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df5b47d15d1426ab3c0ee0bbee6d6d4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:00.626185 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:00.633419 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df5b47d15d1426ab3c0ee0bbee6d6d4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:00.641789 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df5b47d15d1426ab3c0ee0bbee6d6d4'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:00.659487 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1434/2846] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:02:00 2022] DELETE /identity/v3/users/1df5b47d15d1426ab3c0ee0bbee6d6d4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:00.664250 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.664766 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.664881 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.684287 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:00.685527 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:00.750194 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1413/2847] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:00.761535 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:00.763254 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:00.826060 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d5bb-8897-410e-b9e3-1bd056f46667 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:00.830781 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:00.831381 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:00.831653 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:00.831917 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:00.841617 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:00.843156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:00.852686 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1435/2848] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:02:01.132634 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:01.134195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:01.147936 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdb79c-4269-4270-8569-7a7fd144673b None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:01.154309 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:01.154946 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:01.155696 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:01.155943 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] PATH_INFO: `/v3/projects/43210c40bca84b519484faaa25e07875` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:01.157284 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] RBAC: Authorizing `identity:delete_project(project_id=43210c40bca84b519484faaa25e07875)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:01.159207 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:01.211536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43210c40bca84b519484faaa25e07875', 'request_id': 'req-a55e744f-6088-4cfa-9712-d8480ca4371a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.218641 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43210c40bca84b519484faaa25e07875', 'request_id': 'req-a55e744f-6088-4cfa-9712-d8480ca4371a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.225407 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43210c40bca84b519484faaa25e07875', 'request_id': 'req-a55e744f-6088-4cfa-9712-d8480ca4371a'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.239295 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] The token cache is being invalidate because project 43210c40bca84b519484faaa25e07875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:02:01.239575 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.240183 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.241718 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1414/2849] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:02:01 2022] DELETE /identity/v3/projects/43210c40bca84b519484faaa25e07875 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 11:02:01.245835 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:01.255776 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297041-00cd-4803-8ac2-2208dd492613 service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:01.312534 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:01.313150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:01.313424 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:01.313692 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] PATH_INFO: `/v3/users/156024649ac04d8b9269b353f8dcc928` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:01.314843 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=156024649ac04d8b9269b353f8dcc928)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:01.316782 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:01.371411 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156024649ac04d8b9269b353f8dcc928'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.372087 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156024649ac04d8b9269b353f8dcc928'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.372724 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156024649ac04d8b9269b353f8dcc928'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.379608 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156024649ac04d8b9269b353f8dcc928'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.388180 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-a53713ff-2412-41e8-886f-6b48f64ed268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156024649ac04d8b9269b353f8dcc928'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.393654 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:01.409230 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1436/2850] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:02:01 2022] DELETE /identity/v3/users/156024649ac04d8b9269b353f8dcc928 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:01.692799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:01.694476 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:01.708535 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e744f-6088-4cfa-9712-d8480ca4371a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:01.721221 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:01.767964 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:01.768544 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:01.768819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:01.769084 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] PATH_INFO: `/v3/projects/4651f0649d44421aba9e71ef7b698deb` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:01.770205 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4651f0649d44421aba9e71ef7b698deb)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:01.772117 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:01.818727 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4651f0649d44421aba9e71ef7b698deb', 'request_id': 'req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.824252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4651f0649d44421aba9e71ef7b698deb', 'request_id': 'req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.830788 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4651f0649d44421aba9e71ef7b698deb', 'request_id': 'req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.844826 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] The token cache is being invalidate because project 4651f0649d44421aba9e71ef7b698deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:02:01.845108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.845799 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-645f88eb-53fd-443c-b3a9-a5fe8a7e95de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:01.847393 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1415/2851] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:02:01 2022] DELETE /identity/v3/projects/4651f0649d44421aba9e71ef7b698deb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:01.853370 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1afd-5699-4cfe-9b91-a4f7971deeb1 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:01.853923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1afd-5699-4cfe-9b91-a4f7971deeb1 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:01.853923 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1afd-5699-4cfe-9b91-a4f7971deeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:01.874513 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-65cd1afd-5699-4cfe-9b91-a4f7971deeb1 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:01.875746 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-65cd1afd-5699-4cfe-9b91-a4f7971deeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:01.936901 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1437/2852] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:01.945108 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d68724-62d3-4e32-bb42-3c4ca7ee28c9 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:01.945394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d68724-62d3-4e32-bb42-3c4ca7ee28c9 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:01.945667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d68724-62d3-4e32-bb42-3c4ca7ee28c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:01.966620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-40d68724-62d3-4e32-bb42-3c4ca7ee28c9 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:01.967938 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-40d68724-62d3-4e32-bb42-3c4ca7ee28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:01.977600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1416/2853] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:01.981866 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1afd-5699-4cfe-9b91-a4f7971deeb1 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:01.992098 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1afd-5699-4cfe-9b91-a4f7971deeb1 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:01.997190 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:01.997800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:01.998122 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:01.998392 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:01.999037 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 11:02:01.999335 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.002831 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.014990 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1438/2854] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 11:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 11:02:02.019620 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d68724-62d3-4e32-bb42-3c4ca7ee28c9 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.029768 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d68724-62d3-4e32-bb42-3c4ca7ee28c9 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.035865 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.036487 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.036761 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.037283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.037914 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.039821 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.069412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1417/2855] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:02:02 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:02:02.073718 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.084784 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c079-fd2e-4883-b897-6b5c7ff6d2e7 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.142363 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.142992 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.143271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.143537 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.144150 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.146092 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.168693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1439/2856] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:02:02 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:02:02.174817 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.189795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac012df-ff23-4d14-8f96-4e60ff6180dd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.197040 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.197624 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.197870 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.198188 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.198873 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.200490 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.211195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1418/2857] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:02:02.215806 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.228821 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bdea3-7e60-43b4-b70d-9792e034c523 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.233792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5801700-75ca-4067-ae70-1a5d36408b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.234399 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5801700-75ca-4067-ae70-1a5d36408b82 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.234675 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5801700-75ca-4067-ae70-1a5d36408b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.234944 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5801700-75ca-4067-ae70-1a5d36408b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.235596 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5801700-75ca-4067-ae70-1a5d36408b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.237380 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5801700-75ca-4067-ae70-1a5d36408b82 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.248284 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1440/2858] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:02:02.252526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.262898 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c758651-6ca6-4cc1-8a27-32385c19cd99 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.267736 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.268409 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.268667 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.268915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] PATH_INFO: `/v3/projects/3e9e6d3ce3c8452bb0e2309496631311/users/d8b390bd0c3c4ff3af5c4904bb83f056/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.283906 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9e6d3ce3c8452bb0e2309496631311, user_id=d8b390bd0c3c4ff3af5c4904bb83f056, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.286536 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.301672 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1419/2859] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:02:02 2022] PUT /identity/v3/projects/3e9e6d3ce3c8452bb0e2309496631311/users/d8b390bd0c3c4ff3af5c4904bb83f056/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:02.312223 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7793139-41a8-465e-b4f7-9a94ae4d0ef0 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.312531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7793139-41a8-465e-b4f7-9a94ae4d0ef0 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.312797 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7793139-41a8-465e-b4f7-9a94ae4d0ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.343355 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-e7793139-41a8-465e-b4f7-9a94ae4d0ef0 None None] MFA Rules not processed for user `d8b390bd0c3c4ff3af5c4904bb83f056`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:02.344602 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-e7793139-41a8-465e-b4f7-9a94ae4d0ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:02.387895 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:02.413543 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1441/2860] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:02.418560 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.475911 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:02.495289 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686baeb7-5695-430e-b200-2712dfd757e3 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.500452 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.501063 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.501335 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.501605 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.502233 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.504174 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.536807 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1420/2861] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:02:02 2022] POST /identity/v3/projects => generated 401 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:02:02.542750 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7793139-41a8-465e-b4f7-9a94ae4d0ef0 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.556074 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7793139-41a8-465e-b4f7-9a94ae4d0ef0 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.619552 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.620156 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.620432 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.620730 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.621345 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.623276 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.648132 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1442/2862] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:02:02 2022] POST /identity/v3/users => generated 307 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:02:02.653861 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.664626 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f367e-23db-4f33-a2f0-bbde29478bce admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.669088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.669674 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.669947 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.670214 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.670856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.672483 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.683352 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1421/2863] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 11:02:02.687536 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.697933 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b851a-68f9-4d5f-b7d7-2caae3de9514 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.703023 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.703667 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.703932 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.704204 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] PATH_INFO: `/v3/projects/d2388b88d0b64e55b067e5ba95922efa/users/c8b8399cca87469b887f2dc4ecb38c26/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.719959 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2388b88d0b64e55b067e5ba95922efa, user_id=c8b8399cca87469b887f2dc4ecb38c26, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.723088 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.738272 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1443/2864] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:02:02 2022] PUT /identity/v3/projects/d2388b88d0b64e55b067e5ba95922efa/users/c8b8399cca87469b887f2dc4ecb38c26/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 11:02:02.742400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.752903 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81e4d-b70d-4103-b4b2-57450cebec5e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.811545 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b86b6-172f-44a8-b6f3-170bf98c4500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.812096 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b86b6-172f-44a8-b6f3-170bf98c4500 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.812412 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b86b6-172f-44a8-b6f3-170bf98c4500 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.812680 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b86b6-172f-44a8-b6f3-170bf98c4500 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.813324 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b86b6-172f-44a8-b6f3-170bf98c4500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.815005 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b86b6-172f-44a8-b6f3-170bf98c4500 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.824820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1422/2865] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 11:02:02.829174 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.839394 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bd4dc-1c10-4df8-98db-b09f2bc5f9ad admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:02.843950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:02.844629 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] REQUEST_METHOD: `PUT` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.844908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.845233 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] PATH_INFO: `/v3/projects/d2388b88d0b64e55b067e5ba95922efa/users/c8b8399cca87469b887f2dc4ecb38c26/roles/e65d667683404c438483a1b1252ff2bb` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.847117 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2388b88d0b64e55b067e5ba95922efa, user_id=c8b8399cca87469b887f2dc4ecb38c26, role_id=e65d667683404c438483a1b1252ff2bb)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:02.849773 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:02.860064 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1444/2866] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:02:02 2022] PUT /identity/v3/projects/d2388b88d0b64e55b067e5ba95922efa/users/c8b8399cca87469b887f2dc4ecb38c26/roles/e65d667683404c438483a1b1252ff2bb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:02.872252 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0317e-5b85-4287-a7c1-8050257b9325 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:02.872563 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0317e-5b85-4287-a7c1-8050257b9325 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:02.872837 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0317e-5b85-4287-a7c1-8050257b9325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:02.903118 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-d1c0317e-5b85-4287-a7c1-8050257b9325 None None] MFA Rules not processed for user `c8b8399cca87469b887f2dc4ecb38c26`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:02.904357 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-d1c0317e-5b85-4287-a7c1-8050257b9325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:02.974415 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1423/2867] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:02 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:02.985418 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:02.987393 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:03.045529 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a41be-a9b6-4efe-bb8d-2253c9f84673 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:03.049930 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:03.050521 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:03.050792 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:03.051053 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:03.059914 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:03.061373 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:03.071138 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1445/2868] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:02 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:02:03.347697 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0317e-5b85-4287-a7c1-8050257b9325 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:03.358226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0317e-5b85-4287-a7c1-8050257b9325 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:03.362652 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbfa24-ec12-4e1f-a478-fbd4ddb29368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:03.363351 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbfa24-ec12-4e1f-a478-fbd4ddb29368 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:03.363618 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbfa24-ec12-4e1f-a478-fbd4ddb29368 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:03.364018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbfa24-ec12-4e1f-a478-fbd4ddb29368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:03.398283 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:03.424200 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbfa24-ec12-4e1f-a478-fbd4ddb29368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:03.425668 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbfa24-ec12-4e1f-a478-fbd4ddb29368 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:03.434651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1424/2869] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:02:07.571975 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:07.614214 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:07.638046 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303a95-d5ea-4fcd-9827-aaca2c0faaa6 service cinder] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:07.643273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:07.643954 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:07.644225 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:07.644549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] PATH_INFO: `/v3/users/d8b390bd0c3c4ff3af5c4904bb83f056` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:07.645659 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b390bd0c3c4ff3af5c4904bb83f056)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:07.647601 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:07.708716 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b390bd0c3c4ff3af5c4904bb83f056'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:07.709423 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b390bd0c3c4ff3af5c4904bb83f056'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:07.710104 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b390bd0c3c4ff3af5c4904bb83f056'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:07.716882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b390bd0c3c4ff3af5c4904bb83f056'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:07.726649 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b390bd0c3c4ff3af5c4904bb83f056'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:07.744352 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1446/2870] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:02:07 2022] DELETE /identity/v3/users/d8b390bd0c3c4ff3af5c4904bb83f056 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:07.748993 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:07.749480 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:07.749600 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:07.769516 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:07.770737 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:07.831296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1425/2871] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:07.841273 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:07.842777 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:07.917257 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc944fe-e665-4d94-b8f4-a04838ff0cee admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:07.922183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:07.922834 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:07.923129 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:07.923388 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:07.933471 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:07.934950 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:07.944658 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1447/2872] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 11:02:08.229152 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:08.230592 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:08.244394 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c4bb3-525a-4593-adff-33e902d1622f None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:08.249024 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:08.250170 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:08.250482 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:08.250745 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] PATH_INFO: `/v3/projects/3e9e6d3ce3c8452bb0e2309496631311` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:08.252182 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9e6d3ce3c8452bb0e2309496631311)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:08.254099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:08.301515 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9e6d3ce3c8452bb0e2309496631311', 'request_id': 'req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.307701 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9e6d3ce3c8452bb0e2309496631311', 'request_id': 'req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.314195 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9e6d3ce3c8452bb0e2309496631311', 'request_id': 'req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.328160 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] The token cache is being invalidate because project 3e9e6d3ce3c8452bb0e2309496631311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:02:08.328477 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.329099 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.330651 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1426/2873] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:02:08 2022] DELETE /identity/v3/projects/3e9e6d3ce3c8452bb0e2309496631311 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:08.336491 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:08.346994 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa05a-336e-45b0-abaa-cb3c582c41ca service neutron] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:08.407549 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:08.408183 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:08.408530 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:08.408812 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] PATH_INFO: `/v3/users/c8b8399cca87469b887f2dc4ecb38c26` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:08.410108 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b8399cca87469b887f2dc4ecb38c26)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:08.411993 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:08.413445 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:08.467376 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b8399cca87469b887f2dc4ecb38c26'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.468024 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b8399cca87469b887f2dc4ecb38c26'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.468623 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b8399cca87469b887f2dc4ecb38c26'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.475680 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b8399cca87469b887f2dc4ecb38c26'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.485519 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0c740efa-f33e-46da-ac47-32f944db0e78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b8399cca87469b887f2dc4ecb38c26'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.503848 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1448/2874] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:02:08 2022] DELETE /identity/v3/users/c8b8399cca87469b887f2dc4ecb38c26 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:08.792896 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:08.794420 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:08.807998 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fecfc-d5ca-4567-b8fc-3015d2496ab0 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:08.865269 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:08.865816 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:08.866092 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:08.866360 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] PATH_INFO: `/v3/projects/d2388b88d0b64e55b067e5ba95922efa` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:08.867486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2388b88d0b64e55b067e5ba95922efa)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:08.869478 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:08.870915 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:08.919102 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2388b88d0b64e55b067e5ba95922efa', 'request_id': 'req-2b953747-00be-40f0-95e4-74e6c685e699'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.924534 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2388b88d0b64e55b067e5ba95922efa', 'request_id': 'req-2b953747-00be-40f0-95e4-74e6c685e699'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.930504 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2388b88d0b64e55b067e5ba95922efa', 'request_id': 'req-2b953747-00be-40f0-95e4-74e6c685e699'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.945245 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] The token cache is being invalidate because project d2388b88d0b64e55b067e5ba95922efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:02:08.945526 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.946134 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-2b953747-00be-40f0-95e4-74e6c685e699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:08.947598 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1427/2875] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:02:08 2022] DELETE /identity/v3/projects/d2388b88d0b64e55b067e5ba95922efa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:08.952598 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e3944-3568-4249-a4d1-d616480c4e5f None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:08.953079 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e3944-3568-4249-a4d1-d616480c4e5f None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:08.953184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e3944-3568-4249-a4d1-d616480c4e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:08.973800 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-5e2e3944-3568-4249-a4d1-d616480c4e5f None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:08.975000 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-5e2e3944-3568-4249-a4d1-d616480c4e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:09.043647 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1449/2876] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 11:02:09.051822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3781-668b-4002-a7f9-2550b8c10e76 None None] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.052088 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3781-668b-4002-a7f9-2550b8c10e76 None None] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.052424 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3781-668b-4002-a7f9-2550b8c10e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.072944 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.auth.core [None req-095b3781-668b-4002-a7f9-2550b8c10e76 None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:09.074162 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-095b3781-668b-4002-a7f9-2550b8c10e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:09.082733 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1428/2877] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:09.087184 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e3944-3568-4249-a4d1-d616480c4e5f None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:09.098458 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e3944-3568-4249-a4d1-d616480c4e5f None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:09.104769 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:09.105368 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.105636 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.105908 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.106547 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 11:02:09.106839 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:09.108485 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:09.121282 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1450/2878] 10.222.0.23 () {58 vars in 1153 bytes} [Fri Aug 12 11:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 11:02:09.125665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b3781-668b-4002-a7f9-2550b8c10e76 None None] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:09.135631 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b3781-668b-4002-a7f9-2550b8c10e76 None None] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:09.141939 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:09.142539 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] REQUEST_METHOD: `POST` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.142822 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.143163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] PATH_INFO: `/v3/projects` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.143778 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:09.145775 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:09.180163 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1429/2879] 10.222.0.23 () {60 vars in 1153 bytes} [Fri Aug 12 11:02:09 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:02:09.184771 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:09.196693 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2be94-4cea-4c9e-a4f6-72455fe3c8c4 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:09.264402 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:09.265038 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.265332 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.265597 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.266208 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:09.268235 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:09.280081 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:09.295553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1451/2880] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Aug 12 11:02:09 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 11:02:09.299901 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:09.311007 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f68ef-52ca-401e-b3dd-25590175b3cd admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:09.316611 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:09.317192 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.317486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.317748 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.318377 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:09.319966 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:09.332219 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1430/2881] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:02:09.336523 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:09.347527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93378d2-9f79-47b2-ac43-7528d910e4f6 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:09.353329 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3aac5-a47b-4ff4-925a-10a42ffd5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:09.353934 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3aac5-a47b-4ff4-925a-10a42ffd5dad admin admin] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.354205 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3aac5-a47b-4ff4-925a-10a42ffd5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.354464 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3aac5-a47b-4ff4-925a-10a42ffd5dad admin admin] PATH_INFO: `/v3/roles` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.355101 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3aac5-a47b-4ff4-925a-10a42ffd5dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:09.356727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3aac5-a47b-4ff4-925a-10a42ffd5dad admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:09.368760 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1452/2882] 10.222.0.23 () {58 vars in 1122 bytes} [Fri Aug 12 11:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 11:02:09.373904 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:09.385416 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650c59-0f96-4d21-bf6b-2d8e79ff5c21 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:09.390486 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:09.391128 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.391401 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.391670 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] PATH_INFO: `/v3/projects/8fd3c23e31784e97afc14144d3abb3fa/users/3faa78309eb84e44b7cb7f7a54d541a8/roles/b96d83bc5d78463dbf0415bc66a0b0db` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.410019 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd3c23e31784e97afc14144d3abb3fa, user_id=3faa78309eb84e44b7cb7f7a54d541a8, role_id=b96d83bc5d78463dbf0415bc66a0b0db)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:09.412740 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:09.430255 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1431/2883] 10.222.0.23 () {60 vars in 1483 bytes} [Fri Aug 12 11:02:09 2022] PUT /identity/v3/projects/8fd3c23e31784e97afc14144d3abb3fa/users/3faa78309eb84e44b7cb7f7a54d541a8/roles/b96d83bc5d78463dbf0415bc66a0b0db => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:09.442715 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.442997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.443271 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.474126 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] MFA Rules not processed for user `3faa78309eb84e44b7cb7f7a54d541a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:09.475811 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:09.553727 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1453/2884] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:09.565678 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:09.567583 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:09.625889 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a662b5-1518-4f25-9e05-a897d2fb5d5e admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:09.630685 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76f4fb448f1443b7a77fd8e1954defe5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:09.631288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:09.631555 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:09.631819 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:09.640241 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:09.641712 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:09.650386 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1432/2885] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:02:19.550890 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=39 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:19.646647 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:34.575924 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:34.577478 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:34.659981 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76b1b-a8da-4a0a-be98-ffdd4d860e34 None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:34.666816 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:34.667341 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] REQUEST_METHOD: `GET` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:34.667616 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:34.667891 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:34.669791 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:34.686591 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:34.688241 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:34.698665 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1454/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:02:37.653942 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:37.655496 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:37.726969 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d3951-47db-4421-af05-2a5db58d9b84 service cinder] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:37.731744 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:37.732399 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:37.732679 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:37.732950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] PATH_INFO: `/v3/users/f16bdb81c5a249bb9f95e390dbd0dbfd` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:37.734403 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16bdb81c5a249bb9f95e390dbd0dbfd)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:37.736070 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:37.787141 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16bdb81c5a249bb9f95e390dbd0dbfd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:37.794400 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16bdb81c5a249bb9f95e390dbd0dbfd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:37.795098 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16bdb81c5a249bb9f95e390dbd0dbfd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:37.806199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16bdb81c5a249bb9f95e390dbd0dbfd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:37.820244 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16bdb81c5a249bb9f95e390dbd0dbfd'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:37.826018 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1433/2887] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:02:37 2022] DELETE /identity/v3/users/f16bdb81c5a249bb9f95e390dbd0dbfd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:38.024071 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:38.024889 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db56e-bc74-4bef-a712-6eae8c539f23 service placement] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:38.083248 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:38.083798 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:38.084067 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:38.084359 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] PATH_INFO: `/v3/projects/fafc312df1404fc6bbc28415bb744411` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:38.085502 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fafc312df1404fc6bbc28415bb744411)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:38.087401 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:38.134689 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fafc312df1404fc6bbc28415bb744411', 'request_id': 'req-b185fe1e-3812-4d9d-a617-d94f657acf65'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.140861 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fafc312df1404fc6bbc28415bb744411', 'request_id': 'req-b185fe1e-3812-4d9d-a617-d94f657acf65'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.146362 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fafc312df1404fc6bbc28415bb744411', 'request_id': 'req-b185fe1e-3812-4d9d-a617-d94f657acf65'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.161729 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] The token cache is being invalidate because project fafc312df1404fc6bbc28415bb744411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:02:38.162073 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.163195 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.165548 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1455/2888] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:02:38 2022] DELETE /identity/v3/projects/fafc312df1404fc6bbc28415bb744411 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:38.169820 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:38.171266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:38.186770 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab62c5-3bb4-4552-bd86-158139af4a62 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:38.245931 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:38.246551 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:38.246824 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:38.247077 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] PATH_INFO: `/v3/users/0b69cf1ce40e4420ab7000a360d4d158` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:38.248251 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b69cf1ce40e4420ab7000a360d4d158)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:38.250296 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:38.271293 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:38.299358 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b69cf1ce40e4420ab7000a360d4d158'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.306213 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b69cf1ce40e4420ab7000a360d4d158'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.306864 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b69cf1ce40e4420ab7000a360d4d158'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.316081 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b69cf1ce40e4420ab7000a360d4d158'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.327784 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b69cf1ce40e4420ab7000a360d4d158'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.333522 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1434/2889] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:02:38 2022] DELETE /identity/v3/users/0b69cf1ce40e4420ab7000a360d4d158 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:38.515996 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:38.517578 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:38.532489 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185fe1e-3812-4d9d-a617-d94f657acf65 admin admin] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:38.551387 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:38.592939 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:38.593516 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:38.593783 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:38.594047 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] PATH_INFO: `/v3/projects/52bab08340a44d41bd50d25a6b1514f1` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:38.595258 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52bab08340a44d41bd50d25a6b1514f1)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:38.597262 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:38.657140 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52bab08340a44d41bd50d25a6b1514f1', 'request_id': 'req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.663705 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52bab08340a44d41bd50d25a6b1514f1', 'request_id': 'req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.669819 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52bab08340a44d41bd50d25a6b1514f1', 'request_id': 'req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.687374 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] The token cache is being invalidate because project 52bab08340a44d41bd50d25a6b1514f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:02:38.687807 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.688531 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-46b7b32a-27be-4c2e-8f6d-ac493ca3d600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:38.690093 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1456/2890] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:02:38 2022] DELETE /identity/v3/projects/52bab08340a44d41bd50d25a6b1514f1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:45.224950 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:45.226570 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:45.240908 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0306e0-2241-4182-9178-8695abc7ac2a admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:45.301356 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:45.301978 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:45.302248 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:45.302512 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] PATH_INFO: `/v3/users/3faa78309eb84e44b7cb7f7a54d541a8` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:45.303653 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3faa78309eb84e44b7cb7f7a54d541a8)` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:45.305594 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:45.357771 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3faa78309eb84e44b7cb7f7a54d541a8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.365380 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3faa78309eb84e44b7cb7f7a54d541a8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.366035 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3faa78309eb84e44b7cb7f7a54d541a8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.375655 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3faa78309eb84e44b7cb7f7a54d541a8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.388144 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.notifications [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3faa78309eb84e44b7cb7f7a54d541a8'} {{(pid=93138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.394499 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1435/2891] 10.222.0.23 () {58 vars in 1224 bytes} [Fri Aug 12 11:02:45 2022] DELETE /identity/v3/users/3faa78309eb84e44b7cb7f7a54d541a8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:45.399080 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] REQUEST_METHOD: `POST` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:45.399527 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:45.399635 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:45.419997 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.auth.core [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] MFA Rules not processed for user `28302e0508fb4cc195a19e42f7da3046`. Rule list: `[]` (Enabled: `True`). {{(pid=93139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:02:45.421306 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:45.491182 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1457/2892] 10.222.0.23 () {58 vars in 958 bytes} [Fri Aug 12 11:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 11:02:45.501656 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:45.503226 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:45.523664 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:45.572246 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb762e47-b8f4-499d-8271-169f90a61151 admin admin] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:45.576113 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8f3a56366d348bfb0a5c445905dc46b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:45.576755 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:45.577028 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:45.577288 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:45.586657 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:45.588080 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:45.597405 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1436/2893] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:02:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 11:02:45.889512 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] Authenticating user token {{(pid=93139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:02:45.891508 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.fernet_utils [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:02:45.904662 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e6edd-bc4a-4e5f-8329-c9364961759e None None] Validating token access rules against request {{(pid=93139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:02:45.910493 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28302e0508fb4cc195a19e42f7da3046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6e50f80fbe471b845ef361f068a613', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:02:45.911194 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:02:45.911427 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] SCRIPT_NAME: `/identity` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:02:45.911852 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] PATH_INFO: `/v3/projects/8fd3c23e31784e97afc14144d3abb3fa` {{(pid=93139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:02:45.913232 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd3c23e31784e97afc14144d3abb3fa)` {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:02:45.915283 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] RBAC: Authorization granted {{(pid=93139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:02:45.965882 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:45.968709 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd3c23e31784e97afc14144d3abb3fa', 'request_id': 'req-0624298e-9d41-42c6-b4c1-e455d0ba8327'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.977876 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd3c23e31784e97afc14144d3abb3fa', 'request_id': 'req-0624298e-9d41-42c6-b4c1-e455d0ba8327'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.984475 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd3c23e31784e97afc14144d3abb3fa', 'request_id': 'req-0624298e-9d41-42c6-b4c1-e455d0ba8327'} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.998717 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] The token cache is being invalidate because project 8fd3c23e31784e97afc14144d3abb3fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 11:02:45.998942 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:45.999553 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG keystone.notifications [None req-0624298e-9d41-42c6-b4c1-e455d0ba8327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 11:02:46.001120 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: [pid: 93139|app: 0|req: 1458/2894] 10.222.0.23 () {58 vars in 1233 bytes} [Fri Aug 12 11:02:45 2022] DELETE /identity/v3/projects/8fd3c23e31784e97afc14144d3abb3fa => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 11:02:55.593629 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=18 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:02:55.995832 dv-cinder-852945-1-iscsi devstack@keystone.service[93139]: DEBUG dbcounter [-] [93139] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=93139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 11:04:35.776199 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] Authenticating user token {{(pid=93138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:04:35.778339 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:04:35.861855 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f311d-c341-4c1f-aca5-e6c8bb2a340f service neutron] Validating token access rules against request {{(pid=93138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:04:35.929665 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f571bd5-ee7d-4e37-a072-5305b22a7cf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbb12187c53a40e79fe0227256346aeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e93cd13261f41b4971e1b0a6699a850', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:04:35.930266 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f571bd5-ee7d-4e37-a072-5305b22a7cf8 service placement] REQUEST_METHOD: `GET` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:04:35.930548 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f571bd5-ee7d-4e37-a072-5305b22a7cf8 service placement] SCRIPT_NAME: `/identity` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:04:35.930795 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f571bd5-ee7d-4e37-a072-5305b22a7cf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:04:35.932780 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.fernet_utils [None req-4f571bd5-ee7d-4e37-a072-5305b22a7cf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:04:35.944107 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f571bd5-ee7d-4e37-a072-5305b22a7cf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:04:35.945714 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f571bd5-ee7d-4e37-a072-5305b22a7cf8 service placement] RBAC: Authorization granted {{(pid=93138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:04:35.955856 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: [pid: 93138|app: 0|req: 1437/2895] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Aug 12 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 11:04:45.951573 dv-cinder-852945-1-iscsi devstack@keystone.service[93138]: DEBUG dbcounter [-] [93138] Writing DB stats keystone:SELECT=58 {{(pid=93138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}